annotate MoinMoin/action/cache.py @ 6111:1fdd537e9d83

SubProcess: reimplement exec_cmd subclassing Popen and overriding some methods isn't pretty. the code we have was written for py 2.4 or so and the py 2.7 Popen looked quite different. this way with the timer should be less problematic.
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Tue, 06 Sep 2016 04:39:28 +0200
parents 500f68d3e2fd
children 561b7a9c2bd9
rev   line source
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
1 # -*- coding: iso-8859-1 -*-
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
2 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
3 MoinMoin - Send a raw object from the caching system (and offer utility
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
4 functions to put data into cache, calculate cache key, etc.).
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
5
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
6 Sample usage
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
7 ------------
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
8 Assume we have a big picture (bigpic) and we want to efficiently show some
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
9 thumbnail (thumbpic) for it:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
10
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
11 # first calculate a (hard to guess) cache key (this key will change if the
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
12 # original data (bigpic) changes):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
13 key = cache.key(..., attachname=bigpic, ...)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
14
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
15 # check if we don't have it in cache yet
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
16 if not cache.exists(..., key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
17 # if we don't have it in cache, we need to render it - this is an
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
18 # expensive operation that we want to avoid by caching:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
19 thumbpic = render_thumb(bigpic)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
20 # put expensive operation's results into cache:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
21 cache.put(..., key, thumbpic, ...)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
22
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
23 url = cache.url(..., key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
24 html = '<img src="%s">' % url
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
25
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
26 @copyright: 2008 MoinMoin:ThomasWaldmann
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
27 @license: GNU GPL, see COPYING for details.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
28 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
29
5945
d73431c6d2c9 fix 304 http status for cache action, see MoinMoinBugs/CacheActionComparesTimestampsIncorrectly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5591
diff changeset
30 from datetime import datetime
6103
500f68d3e2fd remove our own usage of python_compatibility module
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5945
diff changeset
31 import hmac
5945
d73431c6d2c9 fix 304 http status for cache action, see MoinMoinBugs/CacheActionComparesTimestampsIncorrectly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5591
diff changeset
32
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
33 from MoinMoin import log
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
34 logging = log.getLogger(__name__)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
35
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
36 # keep both imports below as they are, order is important:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
37 from MoinMoin import wikiutil
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
38 import mimetypes
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
39
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
40 from MoinMoin import config, caching
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
41 from MoinMoin.util import filesys
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
42 from MoinMoin.action import AttachFile
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
43
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
44 action_name = __name__.split('.')[-1]
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
45
4424
5ad5753ae311 pre-1.9: request.form has qs args and post data, 1.9: .form only post data, .args only qs args, .values both
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4376
diff changeset
46 # Do NOT get this directly from request.values or user would be able to read any cache!
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
47 cache_arena = 'sendcache' # just using action_name is maybe rather confusing
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
48
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
49 # We maybe could use page local caching (not 'wiki' global) to have less directory entries.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
50 # Local is easier to automatically cleanup if an item changes. Global is easier to manually cleanup.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
51 # Local makes data_dir much larger, harder to backup.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
52 cache_scope = 'wiki'
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
53
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
54 do_locking = False
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
55
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
56 def key(request, wikiname=None, itemname=None, attachname=None, content=None, secret=None):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
57 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
58 Calculate a (hard-to-guess) cache key.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
59
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
60 Important key properties:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
61 * The key must be hard to guess (this is because do=get does no ACL checks,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
62 so whoever got the key [e.g. from html rendering of an ACL protected wiki
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
63 page], will be able to see the cached content.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
64 * The key must change if the (original) content changes. This is because
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
65 ACLs on some item may change and even if somebody was allowed to see some
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
66 revision of some item, it does not implicate that he is allowed to see
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
67 any other revision also. There will be no harm if he can see exactly the
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
68 same content again, but there could be harm if he could access a revision
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
69 with different content.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
70
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
71 If content is supplied, we will calculate and return a hMAC of the content.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
72
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
73 If wikiname, itemname, attachname is given, we don't touch the content (nor do
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
74 we read it ourselves from the attachment file), but we just calculate a key
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
75 from the given metadata values and some metadata we get from the filesystem.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
76
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
77 Hint: if you need multiple cache objects for the same source content (e.g.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
78 thumbnails of different sizes for the same image), calculate the key
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
79 only once and then add some different prefixes to it to get the final
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
80 cache keys.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
81
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
82 @param request: the request object
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
83 @param wikiname: the name of the wiki (if not given, will be read from cfg)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
84 @param itemname: the name of the page
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
85 @param attachname: the filename of the attachment
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
86 @param content: content data as unicode object (e.g. for page content or
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
87 parser section content)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
88 @param secret: secret for hMAC calculation (default: use secret from cfg)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
89 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
90 if secret is None:
3873
e5a9570d3001 secrets configuration refactored - see the snippet for an example usage
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3872
diff changeset
91 secret = request.cfg.secrets['action/cache']
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
92 if content:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
93 hmac_data = content
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
94 elif itemname is not None and attachname is not None:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
95 wikiname = wikiname or request.cfg.interwikiname or request.cfg.siteid
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
96 fuid = filesys.fuid(AttachFile.getFilename(request, itemname, attachname))
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
97 hmac_data = u''.join([wikiname, itemname, attachname, repr(fuid)])
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
98 else:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
99 raise AssertionError('cache_key called with unsupported parameters')
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
100
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
101 hmac_data = hmac_data.encode('utf-8')
6103
500f68d3e2fd remove our own usage of python_compatibility module
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5945
diff changeset
102 key = hmac.new(secret, hmac_data).hexdigest()
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
103 return key
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
104
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
105
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
106 def put(request, key, data,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
107 filename=None,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
108 content_type=None,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
109 content_disposition=None,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
110 content_length=None,
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
111 last_modified=None,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
112 original=None):
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
113 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
114 Put an object into the cache to send it with cache action later.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
115
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
116 @param request: the request object
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
117 @param key: non-guessable key into cache (str)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
118 @param data: content data (str or open file-like obj)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
119 @param filename: filename for content-disposition header and for autodetecting
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
120 content_type (unicode, default: None)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
121 @param content_type: content-type header value (str, default: autodetect from filename)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
122 @param content_disposition: type for content-disposition header (str, default: None)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
123 @param content_length: data length for content-length header (int, default: autodetect)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
124 @param last_modified: last modified timestamp (int, default: autodetect)
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
125 @param original: location of original object (default: None) - this is just written to
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
126 the metadata cache "as is" and could be used for cache cleanup,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
127 use (wikiname, itemname, attachname or None))
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
128 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
129 import os.path
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
130 from MoinMoin.util import timefuncs
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
131
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
132 if filename:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
133 # make sure we just have a simple filename (without path)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
134 filename = os.path.basename(filename)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
135
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
136 if content_type is None:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
137 # try autodetect
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
138 mt, enc = mimetypes.guess_type(filename)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
139 if mt:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
140 content_type = mt
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
141
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
142 if content_type is None:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
143 content_type = 'application/octet-stream'
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
144
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
145 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
146 data_cache.update(data)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
147 content_length = content_length or data_cache.size()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
148 last_modified = last_modified or data_cache.mtime()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
149
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
150 httpdate_last_modified = timefuncs.formathttpdate(int(last_modified))
4578
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
151 headers = [('Content-Type', content_type),
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
152 ('Last-Modified', httpdate_last_modified),
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
153 ('Content-Length', content_length),
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
154 ]
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
155 if content_disposition and filename:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
156 # TODO: fix the encoding here, plain 8 bit is not allowed according to the RFCs
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
157 # There is no solution that is compatible to IE except stripping non-ascii chars
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
158 filename = filename.encode(config.charset)
4578
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
159 headers.append(('Content-Disposition', '%s; filename="%s"' % (content_disposition, filename)))
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
160
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
161 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
162 meta_cache.update({
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
163 'httpdate_last_modified': httpdate_last_modified,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
164 'last_modified': last_modified,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
165 'headers': headers,
3897
78c452cf0257 fix tests, PEP8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3892
diff changeset
166 'original': original,
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
167 })
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
168
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
169
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
170 def exists(request, key, strict=False):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
171 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
172 Check if a cached object for this key exists.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
173
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
174 @param request: the request object
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
175 @param key: non-guessable key into cache (str)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
176 @param strict: if True, also check the data cache, not only meta (bool, default: False)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
177 @return: is object cached? (bool)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
178 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
179 if strict:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
180 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
181 data_cached = data_cache.exists()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
182 else:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
183 data_cached = True # we assume data will be there if meta is there
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
184
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
185 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
186 meta_cached = meta_cache.exists()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
187
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
188 return meta_cached and data_cached
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
189
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
190
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
191 def remove(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
192 """ delete headers/data cache for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
193 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
194 meta_cache.remove()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
195 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
196 data_cache.remove()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
197
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
198
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
199 def url(request, key, do='get'):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
200 """ return URL for the object cached for key """
4252
c2ee4633b9e8 Merged with 1.8
zenhase <zh@punyco.de>
parents: 4235 3903
diff changeset
201 return request.href(action=action_name, do=do, key=key)
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
202
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
203 def _get_headers(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
204 """ get last_modified and headers cached for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
205 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
206 meta = meta_cache.content()
5945
d73431c6d2c9 fix 304 http status for cache action, see MoinMoinBugs/CacheActionComparesTimestampsIncorrectly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5591
diff changeset
207 return meta['last_modified'], meta['headers']
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
208
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
209
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
210 def _get_datafile(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
211 """ get an open data file for the data cached for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
212 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
213 data_cache.open(mode='r')
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
214 return data_cache
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
215
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
216
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
217 def _do_get(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
218 """ send a complete http response with headers/data cached for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
219 try:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
220 last_modified, headers = _get_headers(request, key)
5945
d73431c6d2c9 fix 304 http status for cache action, see MoinMoinBugs/CacheActionComparesTimestampsIncorrectly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5591
diff changeset
221 if datetime.utcfromtimestamp(int(last_modified)) == request.if_modified_since:
4578
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
222 request.status_code = 304
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
223 else:
5591
1dff6cfdcf90 http headers: for most cases, do not use .add, but .__setitem__
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4578
diff changeset
224 for k, v in headers:
1dff6cfdcf90 http headers: for most cases, do not use .add, but .__setitem__
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4578
diff changeset
225 request.headers.add(k, v)
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
226 data_file = _get_datafile(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
227 request.send_file(data_file)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
228 except caching.CacheError:
4578
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
229 request.status_code = 404
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
230
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
231
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
232 def _do_remove(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
233 """ delete headers/data cache for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
234 remove(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
235
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
236
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
237 def _do(request, do, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
238 if do == 'get':
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
239 _do_get(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
240 elif do == 'remove':
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
241 _do_remove(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
242
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
243 def execute(pagename, request):
4424
5ad5753ae311 pre-1.9: request.form has qs args and post data, 1.9: .form only post data, .args only qs args, .values both
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4376
diff changeset
244 do = request.values.get('do')
5ad5753ae311 pre-1.9: request.form has qs args and post data, 1.9: .form only post data, .args only qs args, .values both
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4376
diff changeset
245 key = request.values.get('key')
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
246 _do(request, do, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
247