annotate MoinMoin/support/werkzeug/contrib/cache.py @ 4301:c689dfa55de1

Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
author Florian Krupicka <florian.krupicka@googlemail.com>
date Wed, 13 Aug 2008 21:05:42 +0200
parents
children 246ba4eecab2
rev   line source
4301
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
2 """
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
3 werkzeug.contrib.cache
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
4 ~~~~~~~~~~~~~~~~~~~~~~
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
5
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
6 Small helper module that provides a simple interface to memcached, a
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
7 simple django-inspired in-process cache and a file system based cache.
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
8
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
9 The idea is that it's possible to switch caching systems without changing
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
10 much code in the application.
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
11
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
12
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
13 :copyright: 2007-2008 by Armin Ronacher.
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
14 :license: BSD, see LICENSE for more details.
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
15 """
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
16 import os
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
17 import re
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
18 try:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
19 from hashlib import md5
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
20 except ImportError:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
21 from md5 import new as md5
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
22 from itertools import izip
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
23 from time import time
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
24 from cPickle import loads, dumps, load, dump, HIGHEST_PROTOCOL
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
25
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
26 have_memcache = True
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
27 try:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
28 import cmemcache as memcache
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
29 is_cmemcache = True
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
30 except ImportError:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
31 try:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
32 import memcache
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
33 is_cmemcache = False
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
34 except ImportError:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
35 have_memcache = False
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
36
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
37
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
38 class BaseCache(object):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
39 """Baseclass for the cache systems."""
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
40
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
41 def __init__(self, default_timeout=300):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
42 self.default_timeout = default_timeout
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
43
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
44 def get(self, key):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
45 return None
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
46 delete = get
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
47
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
48 def get_many(self, *keys):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
49 return map(self.get, keys)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
50
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
51 def get_dict(self, *keys):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
52 return dict(izip(keys, self.get_many(keys)))
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
53
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
54 def set(self, key, value, timeout=None):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
55 pass
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
56 add = set
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
57
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
58 def set_many(self, mapping, timeout=None):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
59 for key, value in mapping.iteritems():
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
60 self.set(key, value, timeout)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
61
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
62 def delete_many(self, *keys):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
63 for key in keys:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
64 self.delete(key)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
65
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
66 def clear(self):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
67 pass
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
68
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
69 def inc(self, key, delta=1):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
70 self.set(key, (self.get(key) or 0) + delta)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
71
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
72 def dec(self, key, delta=1):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
73 self.set(key, (self.get(key) or 0) - delta)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
74
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
75
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
76 class NullCache(BaseCache):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
77 """A cache that doesn't cache."""
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
78
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
79
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
80 class SimpleCache(BaseCache):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
81 """Simple memory cache for single process environments. This class exists
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
82 mainly for the development server and is not 100% thread safe. It tries
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
83 to use as many atomic operations as possible and no locks for simplicity
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
84 but it could happen under heavy load that keys are added multiple times.
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
85 """
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
86
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
87 def __init__(self, threshold=500, default_timeout=300):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
88 BaseCache.__init__(self, default_timeout)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
89 self._cache = {}
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
90 self.clear = self._cache.clear
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
91 self._threshold = threshold
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
92
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
93 def _prune(self):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
94 if len(self._cache) > self._threshold:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
95 now = time()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
96 for idx, (key, (expires, _)) in enumerate(self._cache.items()):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
97 if expires <= now or idx % 3 == 0:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
98 self._cache.pop(key, None)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
99
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
100 def get(self, key):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
101 now = time()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
102 expires, value = self._cache.get(key, (0, None))
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
103 if expires > time():
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
104 return loads(value)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
105
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
106 def set(self, key, value, timeout=None):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
107 if timeout is None:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
108 timeout = self.default_timeout
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
109 self._prune()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
110 self._cache[key] = (time() + timeout, dumps(value, HIGHEST_PROTOCOL))
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
111
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
112 def add(self, key, value, timeout=None):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
113 if timeout is None:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
114 timeout = self.default_timeout
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
115 if len(self._cache) > self._threshold:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
116 self._prune()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
117 item = (time() + timeout, dumps(value, HIGHEST_PROTOCOL))
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
118 self._cache.setdefault(key, item)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
119
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
120 def delete(self, key):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
121 self._cache.pop(key, None)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
122
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
123
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
124 _test_memcached_key = re.compile(r'[^\x00-\x21\xff]{1,250}$').match
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
125
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
126 class MemcachedCache(BaseCache):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
127 """A cache that uses memcached as backend.
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
128
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
129 Implementation notes: This cache backend works around some limitations in
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
130 memcached to simplify the interface. For example unicode keys are encoded
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
131 to utf-8 on the fly. Methods such as `get_dict` return the keys in the
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
132 same format as passed. Furthermore all get methods silently ignore key
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
133 errors to not cause problems when untrusted user data is passed to the get
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
134 methods which is often the case in web applications.
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
135 """
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
136
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
137 def __init__(self, servers, default_timeout=300):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
138 BaseCache.__init__(self, default_timeout)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
139 if not have_memcache:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
140 raise RuntimeError('no memcache module found')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
141
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
142 # cmemcache has a bug that debuglog is not defined for the
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
143 # client. Whenever pickle fails you get a weird AttributError.
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
144 if is_cmemcache:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
145 self._client = memcache.Client(map(str, servers))
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
146 try:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
147 self._client.debuglog = lambda *a: None
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
148 except:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
149 pass
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
150 else:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
151 self._client = memcache.Client(servers, False, HIGHEST_PROTOCOL)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
152
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
153 def get(self, key):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
154 if isinstance(key, unicode):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
155 key = key.encode('utf-8')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
156 # memcached doesn't support keys longer than that. Because often
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
157 # checks for so long keys can occour because it's tested from user
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
158 # submitted data etc we fail silently for getting.
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
159 if _test_memcached_key(key):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
160 return self._client.get(key)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
161
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
162 def get_dict(self, *keys):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
163 key_mapping = {}
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
164 have_encoded_keys = False
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
165 for idx, key in enumerate(keys):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
166 if isinstance(key, unicode):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
167 encoded_key = key.encode('utf-8')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
168 have_encoded_keys = True
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
169 else:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
170 encoded_key = key
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
171 if _test_memcached_key(key):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
172 key_mapping[encoded_key] = key
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
173 # the keys call here is important because otherwise cmemcache
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
174 # does ugly things. What exaclty I don't know, i think it does
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
175 # Py_DECREF but quite frankly i don't care.
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
176 d = rv = self._client.get_multi(key_mapping.keys())
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
177 if have_encoded_keys:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
178 rv = {}
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
179 for key, value in d.iteritems():
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
180 rv[key_mapping[key]] = value
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
181 if len(rv) < len(keys):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
182 for key in keys:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
183 if key not in rv:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
184 rv[key] = None
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
185 return rv
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
186
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
187 def add(self, key, value, timeout=None):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
188 if timeout is None:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
189 timeout = self.default_timeout
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
190 if isinstance(key, unicode):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
191 key = key.encode('utf-8')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
192 self._client.add(key, value, timeout)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
193
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
194 def set(self, key, value, timeout=None):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
195 if timeout is None:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
196 timeout = self.default_timeout
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
197 if isinstance(key, unicode):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
198 key = key.encode('utf-8')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
199 self._client.set(key, value, timeout)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
200
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
201 def get_many(self, *keys):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
202 d = self.get_dict(*keys)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
203 return [d[key] for key in keys]
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
204
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
205 def set_many(self, mapping, timeout=None):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
206 if timeout is None:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
207 timeout = self.default_timeout
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
208 new_mapping = {}
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
209 for key, value in mapping.iteritems():
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
210 if isinstance(key, unicode):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
211 key = key.encode('utf-8')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
212 new_mapping[key] = value
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
213 self._client.set_multi(new_mapping, timeout)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
214
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
215 def delete(self, key):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
216 if isinstance(key, unicode):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
217 key = key.encode('utf-8')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
218 self._client.delete(key)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
219
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
220 def delete_many(self, *keys):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
221 keys = list(keys)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
222 for idx, key in enumerate(keys):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
223 if isinstance(key, unicode):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
224 keys[idx] = key.encode('utf-8')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
225 self._client.delete_multi(keys)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
226
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
227 def clear(self):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
228 self._client.flush_all()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
229
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
230 def inc(self, key, delta=1):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
231 if isinstance(key, unicode):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
232 key = key.encode('utf-8')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
233 self._client.incr(key, key, delta)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
234
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
235 def dec(self, key, delta=1):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
236 if isinstance(key, unicode):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
237 key = key.encode('utf-8')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
238 self._client.decr(key, key, delta)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
239
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
240
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
241 class FileSystemCache(BaseCache):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
242 """A cache that stores the items on the file system."""
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
243
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
244 def __init__(self, cache_dir, threshold=500, default_timeout=300):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
245 BaseCache.__init__(self, default_timeout)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
246 self._path = cache_dir
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
247 self._threshold = threshold
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
248 if not os.path.exists(self._path):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
249 os.makedirs(self._path)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
250
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
251 def _prune(self):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
252 entries = os.listdir(self._path)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
253 if len(entries) > self._threshold:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
254 now = time()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
255 for idx, key in enumerate(entries):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
256 try:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
257 f = file(self._get_filename(key))
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
258 if pickle.load(f) > now and idx % 3 != 0:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
259 f.close()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
260 continue
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
261 except:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
262 f.close()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
263 self.delete(key)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
264
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
265 def _get_filename(self, key):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
266 hash = md5(key).hexdigest()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
267 return os.path.join(self._path, hash)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
268
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
269 def get(self, key):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
270 filename = self._get_filename(key)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
271 try:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
272 f = file(filename, 'rb')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
273 try:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
274 if load(f) >= time():
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
275 return load(f)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
276 finally:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
277 f.close()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
278 os.remove(filename)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
279 except:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
280 return None
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
281
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
282 def add(self, key, value, timeout=None):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
283 filename = self._get_filename(key)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
284 if not os.path.exists(filename):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
285 self.set(key, value, timeout)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
286
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
287 def set(self, key, value, timeout=None):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
288 if timeout is None:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
289 timeout = self.default_timeout
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
290 filename = self._get_filename(key)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
291 self._prune()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
292 try:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
293 f = file(filename, 'wb')
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
294 try:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
295 dump(int(time() + timeout), f, 1)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
296 dump(value, f, HIGHEST_PROTOCOL)
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
297 finally:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
298 f.close()
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
299 except (IOError, OSError):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
300 pass
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
301
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
302 def delete(self, key):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
303 try:
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
304 os.remove(self._get_filename(key))
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
305 except (IOError, OSError):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
306 pass
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
307
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
308 def clear(self):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
309 for key in os.listdir(self._path):
c689dfa55de1 Import werkzeug 0.3.1 package into MoinMoin.support (still need to insert CREDITS)
Florian Krupicka <florian.krupicka@googlemail.com>
parents:
diff changeset
310 self.delete(key)