annotate MoinMoin/caching.py @ 6101:316986758258

remove MoinMoin.support.difflib
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Tue, 06 Sep 2016 00:21:08 +0200
parents 7e3e406f21d9
children
rev   line source
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
1 # -*- coding: iso-8859-1 -*-
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
2 """
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
3 MoinMoin caching module
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
4
1918
bb2e053067fb fixing copyright headers: remove umlauts (encoding troubles), make epydoc compatible, reformat
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1898
diff changeset
5 @copyright: 2001-2004 by Juergen Hermann <jh@web.de>,
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
6 2006-2009 MoinMoin:ThomasWaldmann,
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
7 2008 MoinMoin:ThomasPfaff
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
8 @license: GNU GPL, see COPYING for details.
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
9 """
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
10
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
11 import os
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
12 import shutil
1647
c0cb0884f7ec use tempfile.mkstemp to avoid renaming trouble (ported from 1.5)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1638
diff changeset
13 import tempfile
1316
c4f6af5ee295 Mute the tempnam warning in the caching module.
Alexander Schremmer <alex AT alexanderweb DOT de>
parents: 1309
diff changeset
14
3127
ea5383222f2f refactored misc. modules to use own logger instead of request.log
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2841
diff changeset
15 from MoinMoin import log
ea5383222f2f refactored misc. modules to use own logger instead of request.log
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2841
diff changeset
16 logging = log.getLogger(__name__)
ea5383222f2f refactored misc. modules to use own logger instead of request.log
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2841
diff changeset
17
1898
b5e94fec8669 caching: mkstemp uses 0600 to create files, we need chmod to change them to our preferred file mode
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1793
diff changeset
18 from MoinMoin import config
1793
2a4caa295346 Eclipse PyDev Check: fixed lots of its errors and warnings
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1791
diff changeset
19 from MoinMoin.util import filesys, lock, pickle, PICKLE_PROTOCOL
931
141083b64fb1 whitespace and minor style changes only
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 902
diff changeset
20
1316
c4f6af5ee295 Mute the tempnam warning in the caching module.
Alexander Schremmer <alex AT alexanderweb DOT de>
parents: 1309
diff changeset
21
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
22 class CacheError(Exception):
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
23 """ raised if we have trouble locking, reading or writing """
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
24 pass
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
25
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
26
1936
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
27 def get_arena_dir(request, arena, scope):
3932
2bcd23157684 caching: remove page_or_wiki scope
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 3879
diff changeset
28 if scope == 'item': # arena is a Page instance
1936
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
29 # we could move cache out of the page directory and store it to cache_dir
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
30 return arena.getPagePath('cache', check_create=1)
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
31 elif scope == 'wiki':
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
32 return os.path.join(request.cfg.cache_dir, request.cfg.siteid, arena)
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
33 elif scope == 'farm':
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
34 return os.path.join(request.cfg.cache_dir, '__common__', arena)
6023
7e3e406f21d9 introduce a new caching scope "userdir"
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5609
diff changeset
35 elif scope == 'userdir':
7e3e406f21d9 introduce a new caching scope "userdir"
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5609
diff changeset
36 return os.path.join(request.cfg.user_dir, 'cache', arena)
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
37 elif scope == 'dir':
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
38 # arena is a specific directory, just use it
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
39 return arena
1936
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
40 return None
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
41
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
42
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
43 def get_cache_list(request, arena, scope):
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
44 arena_dir = get_arena_dir(request, arena, scope)
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
45 try:
5609
4c6a4adf0540 remove all calls to filesys.dc* (dclistdir, dcdisable), remove unused imports
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5142
diff changeset
46 return os.listdir(arena_dir)
1936
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
47 except OSError:
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
48 return []
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
49
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
50
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
51 class CacheEntry:
3932
2bcd23157684 caching: remove page_or_wiki scope
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 3879
diff changeset
52 def __init__(self, request, arena, key, scope='wiki', do_locking=True,
2238
529a3c5e6d0e add use_encode parameter to caching - useful for readable cache files. this re-adds similar functionality that was removed in changeset 5070d0358544
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1936
diff changeset
53 use_pickle=False, use_encode=False):
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
54 """ init a cache entry
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
55 @param request: the request object
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
56 @param arena: either a string or a page object, when we want to use
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
57 page local cache area
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
58 @param key: under which key we access the cache content
681
2d5883e4b82c scope == {'item', 'wiki', 'farm'} for caching
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 680
diff changeset
59 @param scope: the scope where we are caching:
2d5883e4b82c scope == {'item', 'wiki', 'farm'} for caching
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 680
diff changeset
60 'item' - an item local cache
2d5883e4b82c scope == {'item', 'wiki', 'farm'} for caching
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 680
diff changeset
61 'wiki' - a wiki local cache
2d5883e4b82c scope == {'item', 'wiki', 'farm'} for caching
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 680
diff changeset
62 'farm' - a cache for the whole farm
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
63 'dir' - just use some specific directory
2238
529a3c5e6d0e add use_encode parameter to caching - useful for readable cache files. this re-adds similar functionality that was removed in changeset 5070d0358544
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1936
diff changeset
64 @param do_locking: if there should be a lock, normally True
529a3c5e6d0e add use_encode parameter to caching - useful for readable cache files. this re-adds similar functionality that was removed in changeset 5070d0358544
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1936
diff changeset
65 @param use_pickle: if data should be pickled/unpickled (nice for arbitrary cache content)
529a3c5e6d0e add use_encode parameter to caching - useful for readable cache files. this re-adds similar functionality that was removed in changeset 5070d0358544
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1936
diff changeset
66 @param use_encode: if data should be encoded/decoded (nice for readable cache files)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
67 """
412
481c432b3d11 fix some error handlers, logging for caching lock failures
Thomas Waldmann <tw@waldmann-edv.de>
parents: 402
diff changeset
68 self.request = request
1307
6bc162aba29d caching: update contents by tmpfile+rename, not touching old inode content. add locking for .remove() call. small fixes to lock code.
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 931
diff changeset
69 self.key = key
6bc162aba29d caching: update contents by tmpfile+rename, not touching old inode content. add locking for .remove() call. small fixes to lock code.
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 931
diff changeset
70 self.locking = do_locking
1638
5070d0358544 caching: move use_pickle to __init__, remove unused encode/decode args from .content()/.update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1637
diff changeset
71 self.use_pickle = use_pickle
2238
529a3c5e6d0e add use_encode parameter to caching - useful for readable cache files. this re-adds similar functionality that was removed in changeset 5070d0358544
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1936
diff changeset
72 self.use_encode = use_encode
1936
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
73 self.arena_dir = get_arena_dir(request, arena, scope)
1429
1d55eff67af6 check for non-existing dir before calling os.makedirs
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1428
diff changeset
74 if not os.path.exists(self.arena_dir):
1428
8a89649ab5f0 remove util.filesys.make[dD]irs - not needed any more due to os.umask() use
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1317
diff changeset
75 os.makedirs(self.arena_dir)
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
76 self._fname = os.path.join(self.arena_dir, key)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
77
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
78 # used by file-like api:
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
79 self._lock = None # either a read or a write lock
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
80 self._fileobj = None # open cache file object
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
81 self._tmp_fname = None # name of temporary file (used for write)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
82 self._mode = None # mode of open file object
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
83
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
84
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
85 def _filename(self):
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
86 # DEPRECATED - please use file-like api
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
87 return self._fname
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
88
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
89 def exists(self):
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
90 return os.path.exists(self._fname)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
91
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
92 def mtime(self):
3610
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
93 # DEPRECATED for checking a changed on-disk cache, please use
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
94 # self.uid() for this, see below
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
95 try:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
96 return os.path.getmtime(self._fname)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
97 except (IOError, OSError):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
98 return 0
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
99
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
100 def size(self):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
101 try:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
102 return os.path.getsize(self._fname)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
103 except (IOError, OSError):
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
104 return 0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
105
3610
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
106 def uid(self):
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
107 """ Return a value that likely changes when the on-disk cache was updated.
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
108
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
109 See docstring of MoinMoin.util.filesys.fuid for details.
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
110 """
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
111 return filesys.fuid(self._fname)
3610
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
112
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
113 def needsUpdate(self, filename, attachdir=None):
1681
fc03cd8ff0f3 avoid accessing invalid page revisions, tune caching code (ported from 1.5)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1647
diff changeset
114 # following code is not necessary. will trigger exception and give same result
fc03cd8ff0f3 avoid accessing invalid page revisions, tune caching code (ported from 1.5)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1647
diff changeset
115 #if not self.exists():
fc03cd8ff0f3 avoid accessing invalid page revisions, tune caching code (ported from 1.5)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1647
diff changeset
116 # return 1
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
117
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
118 try:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
119 ctime = os.path.getmtime(self._fname)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
120 ftime = os.path.getmtime(filename)
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
121 except os.error:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
122 return 1
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
123
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
124 needsupdate = ftime > ctime
931
141083b64fb1 whitespace and minor style changes only
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 902
diff changeset
125
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
126 # if a page depends on the attachment dir, we check this, too:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
127 if not needsupdate and attachdir:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
128 try:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
129 ftime2 = os.path.getmtime(attachdir)
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
130 except os.error:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
131 ftime2 = 0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
132 needsupdate = ftime2 > ctime
931
141083b64fb1 whitespace and minor style changes only
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 902
diff changeset
133
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
134 return needsupdate
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
135
5142
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
136 def lock(self, mode, timeout=10.0):
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
137 """
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
138 acquire a lock for <mode> ("r" or "w").
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
139 we just raise a CacheError if this doesn't work.
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
140
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
141 Note:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
142 * .open() calls .lock(), .close() calls .unlock() if do_locking is True.
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
143 * if you need to do a read-modify-write, you want to use a CacheEntry
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
144 with do_locking=False and manually call .lock('w') and .unlock().
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
145 """
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
146 lock_dir = os.path.join(self.arena_dir, '__lock__')
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
147 if 'r' in mode:
5142
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
148 _lock = lock.LazyReadLock(lock_dir, 60.0)
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
149 elif 'w' in mode:
5142
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
150 _lock = lock.LazyWriteLock(lock_dir, 60.0)
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
151 acquired = _lock.acquire(timeout)
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
152 if acquired:
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
153 self._lock = _lock
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
154 else:
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
155 self._lock = None
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
156 err = "Can't acquire %s lock in %s" % (mode, lock_dir)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
157 logging.error(err)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
158 raise CacheError(err)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
159
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
160 def unlock(self):
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
161 """
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
162 release the lock.
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
163 """
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
164 if self._lock:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
165 self._lock.release()
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
166 self._lock = None
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
167
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
168 # file-like interface ----------------------------------------------------
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
169
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
170 def open(self, filename=None, mode='r', bufsize=-1):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
171 """ open the cache for reading/writing
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
172
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
173 Typical usage:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
174 try:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
175 cache.open('r') # open file, create locks
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
176 data = cache.read()
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
177 finally:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
178 cache.close() # important to close file and remove locks
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
179
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
180 @param filename: must be None (default - automatically determine filename)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
181 @param mode: 'r' (read, default), 'w' (write)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
182 Note: if mode does not include 'b' (binary), it will be
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
183 automatically changed to include 'b'.
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
184 @param bufsize: size of read/write buffer (default: -1 meaning automatic)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
185 @return: None (the opened file object is kept in self._fileobj and used
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
186 implicitely by read/write/close functions of CacheEntry object.
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
187 """
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
188 assert self._fileobj is None, 'caching: trying to open an already opened cache'
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
189 assert filename is None, 'caching: giving a filename is not supported (yet?)'
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
190 assert 'r' in mode or 'w' in mode, 'caching: mode must contain "r" or "w"'
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
191
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
192 if 'b' not in mode:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
193 mode += 'b' # we want to use binary mode, ever!
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
194 self._mode = mode # for self.close()
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
195
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
196 if self.locking:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
197 self.lock(mode)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
198 try:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
199 if 'r' in mode:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
200 filename = self._fname
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
201 self._fileobj = open(filename, mode, bufsize)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
202 elif 'w' in mode:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
203 # we do not write content to old inode, but to a new file
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
204 # so we don't need to lock when we just want to read the file
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
205 # (at least on POSIX, this works)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
206 filename = None
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
207 fd, filename = tempfile.mkstemp('.tmp', self.key, self.arena_dir)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
208 self._tmp_fname = filename
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
209 self._fileobj = os.fdopen(fd, mode, bufsize)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
210 except IOError, err:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
211 if 'w' in mode:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
212 # IOerror for 'r' can be just a non-existing file, do not log that,
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
213 # but if open fails for 'w', we likely have some bigger problem:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
214 logging.error(str(err))
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
215 raise CacheError(str(err))
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
216
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
217 def read(self, size=-1):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
218 """ read data from cache file
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
219
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
220 @param size: how many bytes to read (default: -1 == everything)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
221 @return: read data (str)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
222 """
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
223 return self._fileobj.read(size)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
224
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
225 def write(self, data):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
226 """ write data to cache file
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
227
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
228 @param data: write data (str)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
229 """
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
230 self._fileobj.write(data)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
231
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
232 def close(self):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
233 """ close cache file (and release lock, if any) """
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
234 try:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
235 if self._fileobj:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
236 self._fileobj.close()
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
237 self._fileobj = None
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
238 if 'w' in self._mode:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
239 filesys.chmod(self._tmp_fname, 0666 & config.umask) # fix mode that mkstemp chose
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
240 # this is either atomic or happening with real locks set:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
241 filesys.rename(self._tmp_fname, self._fname)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
242 finally:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
243 if self.locking:
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
244 self.unlock()
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
245
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
246 # ------------------------------------------------------------------------
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
247
1638
5070d0358544 caching: move use_pickle to __init__, remove unused encode/decode args from .content()/.update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1637
diff changeset
248 def update(self, content):
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
249 try:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
250 if hasattr(content, 'read'):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
251 # content is file-like
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
252 assert not (self.use_pickle or self.use_encode), 'caching: use_pickle and use_encode not supported with file-like api'
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
253 try:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
254 self.open(mode='w')
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
255 shutil.copyfileobj(content, self)
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
256 finally:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
257 self.close()
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
258 else:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
259 # content is a string
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
260 if self.use_pickle:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
261 content = pickle.dumps(content, PICKLE_PROTOCOL)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
262 elif self.use_encode:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
263 content = content.encode(config.charset)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
264
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
265 try:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
266 self.open(mode='w')
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
267 self.write(content)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
268 finally:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
269 self.close()
2841
93be75db2051 handle out-of-space errors in PageEditor and caching, avoid 'current' corruption (thanks to TheAnarcat)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2238
diff changeset
270 except (pickle.PicklingError, OSError, IOError, ValueError), err:
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
271 raise CacheError(str(err))
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
272
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
273 def content(self):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
274 # no file-like api yet, we implement it when we need it
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
275 try:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
276 try:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
277 self.open(mode='r')
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
278 data = self.read()
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
279 finally:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
280 self.close()
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
281 if self.use_pickle:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
282 data = pickle.loads(data)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
283 elif self.use_encode:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
284 data = data.decode(config.charset)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
285 return data
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
286 except (pickle.UnpicklingError, IOError, EOFError, ValueError), err:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
287 raise CacheError(str(err))
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
288
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
289 def remove(self):
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
290 if self.locking:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
291 self.lock('w')
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
292 try:
1307
6bc162aba29d caching: update contents by tmpfile+rename, not touching old inode content. add locking for .remove() call. small fixes to lock code.
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 931
diff changeset
293 try:
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
294 os.remove(self._fname)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
295 except OSError:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
296 pass
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
297 finally:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
298 if self.locking:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
299 self.unlock()
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
300
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
301