annotate MoinMoin/caching.py @ 5142:294b97b991d3 1.8.5

work around windows caching lock problems (see details below) caching: fix wrong releasing of lock (if lock acquiring failed) caching: increase the timeout for trying to acquire a lock (1s -> 10s) filesys: increase retries for filesystem operations on win32 from 10 to 42 The underlying reason that makes these ugly workarounds in filesys necessary so it works on win32 is still unknown.
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Thu, 10 Sep 2009 11:57:44 +0100
parents 95a4aa0eb1e9
children 4c6a4adf0540
rev   line source
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
1 # -*- coding: iso-8859-1 -*-
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
2 """
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
3 MoinMoin caching module
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
4
1918
bb2e053067fb fixing copyright headers: remove umlauts (encoding troubles), make epydoc compatible, reformat
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1898
diff changeset
5 @copyright: 2001-2004 by Juergen Hermann <jh@web.de>,
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
6 2006-2009 MoinMoin:ThomasWaldmann,
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
7 2008 MoinMoin:ThomasPfaff
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
8 @license: GNU GPL, see COPYING for details.
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
9 """
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
10
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
11 import os
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
12 import shutil
1647
c0cb0884f7ec use tempfile.mkstemp to avoid renaming trouble (ported from 1.5)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1638
diff changeset
13 import tempfile
1316
c4f6af5ee295 Mute the tempnam warning in the caching module.
Alexander Schremmer <alex AT alexanderweb DOT de>
parents: 1309
diff changeset
14
3127
ea5383222f2f refactored misc. modules to use own logger instead of request.log
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2841
diff changeset
15 from MoinMoin import log
ea5383222f2f refactored misc. modules to use own logger instead of request.log
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2841
diff changeset
16 logging = log.getLogger(__name__)
ea5383222f2f refactored misc. modules to use own logger instead of request.log
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2841
diff changeset
17
1898
b5e94fec8669 caching: mkstemp uses 0600 to create files, we need chmod to change them to our preferred file mode
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1793
diff changeset
18 from MoinMoin import config
1793
2a4caa295346 Eclipse PyDev Check: fixed lots of its errors and warnings
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1791
diff changeset
19 from MoinMoin.util import filesys, lock, pickle, PICKLE_PROTOCOL
931
141083b64fb1 whitespace and minor style changes only
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 902
diff changeset
20
1316
c4f6af5ee295 Mute the tempnam warning in the caching module.
Alexander Schremmer <alex AT alexanderweb DOT de>
parents: 1309
diff changeset
21
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
22 class CacheError(Exception):
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
23 """ raised if we have trouble locking, reading or writing """
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
24 pass
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
25
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
26
1936
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
27 def get_arena_dir(request, arena, scope):
3932
2bcd23157684 caching: remove page_or_wiki scope
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 3879
diff changeset
28 if scope == 'item': # arena is a Page instance
1936
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
29 # we could move cache out of the page directory and store it to cache_dir
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
30 return arena.getPagePath('cache', check_create=1)
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
31 elif scope == 'wiki':
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
32 return os.path.join(request.cfg.cache_dir, request.cfg.siteid, arena)
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
33 elif scope == 'farm':
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
34 return os.path.join(request.cfg.cache_dir, '__common__', arena)
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
35 elif scope == 'dir':
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
36 # arena is a specific directory, just use it
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
37 return arena
1936
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
38 return None
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
39
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
40
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
41 def get_cache_list(request, arena, scope):
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
42 arena_dir = get_arena_dir(request, arena, scope)
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
43 try:
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
44 return filesys.dclistdir(arena_dir)
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
45 except OSError:
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
46 return []
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
47
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
48
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
49 class CacheEntry:
3932
2bcd23157684 caching: remove page_or_wiki scope
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 3879
diff changeset
50 def __init__(self, request, arena, key, scope='wiki', do_locking=True,
2238
529a3c5e6d0e add use_encode parameter to caching - useful for readable cache files. this re-adds similar functionality that was removed in changeset 5070d0358544
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1936
diff changeset
51 use_pickle=False, use_encode=False):
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
52 """ init a cache entry
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
53 @param request: the request object
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
54 @param arena: either a string or a page object, when we want to use
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
55 page local cache area
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
56 @param key: under which key we access the cache content
681
2d5883e4b82c scope == {'item', 'wiki', 'farm'} for caching
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 680
diff changeset
57 @param scope: the scope where we are caching:
2d5883e4b82c scope == {'item', 'wiki', 'farm'} for caching
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 680
diff changeset
58 'item' - an item local cache
2d5883e4b82c scope == {'item', 'wiki', 'farm'} for caching
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 680
diff changeset
59 'wiki' - a wiki local cache
2d5883e4b82c scope == {'item', 'wiki', 'farm'} for caching
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 680
diff changeset
60 'farm' - a cache for the whole farm
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
61 'dir' - just use some specific directory
2238
529a3c5e6d0e add use_encode parameter to caching - useful for readable cache files. this re-adds similar functionality that was removed in changeset 5070d0358544
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1936
diff changeset
62 @param do_locking: if there should be a lock, normally True
529a3c5e6d0e add use_encode parameter to caching - useful for readable cache files. this re-adds similar functionality that was removed in changeset 5070d0358544
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1936
diff changeset
63 @param use_pickle: if data should be pickled/unpickled (nice for arbitrary cache content)
529a3c5e6d0e add use_encode parameter to caching - useful for readable cache files. this re-adds similar functionality that was removed in changeset 5070d0358544
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1936
diff changeset
64 @param use_encode: if data should be encoded/decoded (nice for readable cache files)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
65 """
412
481c432b3d11 fix some error handlers, logging for caching lock failures
Thomas Waldmann <tw@waldmann-edv.de>
parents: 402
diff changeset
66 self.request = request
1307
6bc162aba29d caching: update contents by tmpfile+rename, not touching old inode content. add locking for .remove() call. small fixes to lock code.
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 931
diff changeset
67 self.key = key
6bc162aba29d caching: update contents by tmpfile+rename, not touching old inode content. add locking for .remove() call. small fixes to lock code.
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 931
diff changeset
68 self.locking = do_locking
1638
5070d0358544 caching: move use_pickle to __init__, remove unused encode/decode args from .content()/.update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1637
diff changeset
69 self.use_pickle = use_pickle
2238
529a3c5e6d0e add use_encode parameter to caching - useful for readable cache files. this re-adds similar functionality that was removed in changeset 5070d0358544
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1936
diff changeset
70 self.use_encode = use_encode
1936
adcdb9cc9258 caching.get_cache_list
Johannes Berg <johannes AT sipsolutions DOT net>
parents: 1920
diff changeset
71 self.arena_dir = get_arena_dir(request, arena, scope)
1429
1d55eff67af6 check for non-existing dir before calling os.makedirs
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1428
diff changeset
72 if not os.path.exists(self.arena_dir):
1428
8a89649ab5f0 remove util.filesys.make[dD]irs - not needed any more due to os.umask() use
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1317
diff changeset
73 os.makedirs(self.arena_dir)
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
74 self._fname = os.path.join(self.arena_dir, key)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
75
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
76 # used by file-like api:
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
77 self._lock = None # either a read or a write lock
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
78 self._fileobj = None # open cache file object
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
79 self._tmp_fname = None # name of temporary file (used for write)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
80 self._mode = None # mode of open file object
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
81
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
82
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
83 def _filename(self):
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
84 # DEPRECATED - please use file-like api
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
85 return self._fname
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
86
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
87 def exists(self):
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
88 return os.path.exists(self._fname)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
89
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
90 def mtime(self):
3610
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
91 # DEPRECATED for checking a changed on-disk cache, please use
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
92 # self.uid() for this, see below
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
93 try:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
94 return os.path.getmtime(self._fname)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
95 except (IOError, OSError):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
96 return 0
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
97
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
98 def size(self):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
99 try:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
100 return os.path.getsize(self._fname)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
101 except (IOError, OSError):
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
102 return 0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
103
3610
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
104 def uid(self):
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
105 """ Return a value that likely changes when the on-disk cache was updated.
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
106
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
107 See docstring of MoinMoin.util.filesys.fuid for details.
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
108 """
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
109 return filesys.fuid(self._fname)
3610
39456a5dc4e8 fix failure of detection of on-disk cache updates (this fixes unreliable dicts/groups tests), remove unused Page attribute
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3127
diff changeset
110
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
111 def needsUpdate(self, filename, attachdir=None):
1681
fc03cd8ff0f3 avoid accessing invalid page revisions, tune caching code (ported from 1.5)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1647
diff changeset
112 # following code is not necessary. will trigger exception and give same result
fc03cd8ff0f3 avoid accessing invalid page revisions, tune caching code (ported from 1.5)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1647
diff changeset
113 #if not self.exists():
fc03cd8ff0f3 avoid accessing invalid page revisions, tune caching code (ported from 1.5)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1647
diff changeset
114 # return 1
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
115
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
116 try:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
117 ctime = os.path.getmtime(self._fname)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
118 ftime = os.path.getmtime(filename)
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
119 except os.error:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
120 return 1
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
121
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
122 needsupdate = ftime > ctime
931
141083b64fb1 whitespace and minor style changes only
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 902
diff changeset
123
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
124 # if a page depends on the attachment dir, we check this, too:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
125 if not needsupdate and attachdir:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
126 try:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
127 ftime2 = os.path.getmtime(attachdir)
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
128 except os.error:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
129 ftime2 = 0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
130 needsupdate = ftime2 > ctime
931
141083b64fb1 whitespace and minor style changes only
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 902
diff changeset
131
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
132 return needsupdate
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
133
5142
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
134 def lock(self, mode, timeout=10.0):
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
135 """
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
136 acquire a lock for <mode> ("r" or "w").
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
137 we just raise a CacheError if this doesn't work.
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
138
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
139 Note:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
140 * .open() calls .lock(), .close() calls .unlock() if do_locking is True.
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
141 * if you need to do a read-modify-write, you want to use a CacheEntry
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
142 with do_locking=False and manually call .lock('w') and .unlock().
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
143 """
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
144 lock_dir = os.path.join(self.arena_dir, '__lock__')
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
145 if 'r' in mode:
5142
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
146 _lock = lock.LazyReadLock(lock_dir, 60.0)
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
147 elif 'w' in mode:
5142
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
148 _lock = lock.LazyWriteLock(lock_dir, 60.0)
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
149 acquired = _lock.acquire(timeout)
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
150 if acquired:
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
151 self._lock = _lock
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
152 else:
294b97b991d3 work around windows caching lock problems (see details below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5051
diff changeset
153 self._lock = None
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
154 err = "Can't acquire %s lock in %s" % (mode, lock_dir)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
155 logging.error(err)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
156 raise CacheError(err)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
157
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
158 def unlock(self):
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
159 """
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
160 release the lock.
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
161 """
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
162 if self._lock:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
163 self._lock.release()
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
164 self._lock = None
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
165
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
166 # file-like interface ----------------------------------------------------
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
167
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
168 def open(self, filename=None, mode='r', bufsize=-1):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
169 """ open the cache for reading/writing
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
170
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
171 Typical usage:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
172 try:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
173 cache.open('r') # open file, create locks
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
174 data = cache.read()
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
175 finally:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
176 cache.close() # important to close file and remove locks
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
177
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
178 @param filename: must be None (default - automatically determine filename)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
179 @param mode: 'r' (read, default), 'w' (write)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
180 Note: if mode does not include 'b' (binary), it will be
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
181 automatically changed to include 'b'.
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
182 @param bufsize: size of read/write buffer (default: -1 meaning automatic)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
183 @return: None (the opened file object is kept in self._fileobj and used
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
184 implicitely by read/write/close functions of CacheEntry object.
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
185 """
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
186 assert self._fileobj is None, 'caching: trying to open an already opened cache'
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
187 assert filename is None, 'caching: giving a filename is not supported (yet?)'
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
188 assert 'r' in mode or 'w' in mode, 'caching: mode must contain "r" or "w"'
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
189
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
190 if 'b' not in mode:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
191 mode += 'b' # we want to use binary mode, ever!
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
192 self._mode = mode # for self.close()
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
193
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
194 if self.locking:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
195 self.lock(mode)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
196 try:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
197 if 'r' in mode:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
198 filename = self._fname
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
199 self._fileobj = open(filename, mode, bufsize)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
200 elif 'w' in mode:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
201 # we do not write content to old inode, but to a new file
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
202 # so we don't need to lock when we just want to read the file
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
203 # (at least on POSIX, this works)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
204 filename = None
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
205 fd, filename = tempfile.mkstemp('.tmp', self.key, self.arena_dir)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
206 self._tmp_fname = filename
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
207 self._fileobj = os.fdopen(fd, mode, bufsize)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
208 except IOError, err:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
209 if 'w' in mode:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
210 # IOerror for 'r' can be just a non-existing file, do not log that,
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
211 # but if open fails for 'w', we likely have some bigger problem:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
212 logging.error(str(err))
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
213 raise CacheError(str(err))
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
214
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
215 def read(self, size=-1):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
216 """ read data from cache file
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
217
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
218 @param size: how many bytes to read (default: -1 == everything)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
219 @return: read data (str)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
220 """
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
221 return self._fileobj.read(size)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
222
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
223 def write(self, data):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
224 """ write data to cache file
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
225
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
226 @param data: write data (str)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
227 """
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
228 self._fileobj.write(data)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
229
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
230 def close(self):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
231 """ close cache file (and release lock, if any) """
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
232 try:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
233 if self._fileobj:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
234 self._fileobj.close()
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
235 self._fileobj = None
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
236 if 'w' in self._mode:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
237 filesys.chmod(self._tmp_fname, 0666 & config.umask) # fix mode that mkstemp chose
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
238 # this is either atomic or happening with real locks set:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
239 filesys.rename(self._tmp_fname, self._fname)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
240 finally:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
241 if self.locking:
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
242 self.unlock()
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
243
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
244 # ------------------------------------------------------------------------
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
245
1638
5070d0358544 caching: move use_pickle to __init__, remove unused encode/decode args from .content()/.update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1637
diff changeset
246 def update(self, content):
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
247 try:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
248 if hasattr(content, 'read'):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
249 # content is file-like
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
250 assert not (self.use_pickle or self.use_encode), 'caching: use_pickle and use_encode not supported with file-like api'
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
251 try:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
252 self.open(mode='w')
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
253 shutil.copyfileobj(content, self)
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
254 finally:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
255 self.close()
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
256 else:
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
257 # content is a string
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
258 if self.use_pickle:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
259 content = pickle.dumps(content, PICKLE_PROTOCOL)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
260 elif self.use_encode:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
261 content = content.encode(config.charset)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
262
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
263 try:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
264 self.open(mode='w')
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
265 self.write(content)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
266 finally:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
267 self.close()
2841
93be75db2051 handle out-of-space errors in PageEditor and caching, avoid 'current' corruption (thanks to TheAnarcat)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2238
diff changeset
268 except (pickle.PicklingError, OSError, IOError, ValueError), err:
1637
47a173ae5401 caching module: use_pickle argument for .content() and .update()
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1429
diff changeset
269 raise CacheError(str(err))
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
270
3878
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
271 def content(self):
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
272 # no file-like api yet, we implement it when we need it
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
273 try:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
274 try:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
275 self.open(mode='r')
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
276 data = self.read()
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
277 finally:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
278 self.close()
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
279 if self.use_pickle:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
280 data = pickle.loads(data)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
281 elif self.use_encode:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
282 data = data.decode(config.charset)
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
283 return data
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
284 except (pickle.UnpicklingError, IOError, EOFError, ValueError), err:
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
285 raise CacheError(str(err))
b13a58a18dac caching module backported from 1.8 (file-like new api, old api still as is)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3610
diff changeset
286
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
287 def remove(self):
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
288 if self.locking:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
289 self.lock('w')
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
290 try:
1307
6bc162aba29d caching: update contents by tmpfile+rename, not touching old inode content. add locking for .remove() call. small fixes to lock code.
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 931
diff changeset
291 try:
5051
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
292 os.remove(self._fname)
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
293 except OSError:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
294 pass
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
295 finally:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
296 if self.locking:
95a4aa0eb1e9 caching: refactored locking to separate methods, new scope='dir'
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5049
diff changeset
297 self.unlock()
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
298
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
299