changeset 811:d7b0946924f9 pytest2

improve indexing admin docs
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Wed, 31 Aug 2011 14:48:53 +0200
parents 449c081c2a50
children 39fc3b341002
files docs/admin/index.rst
diffstat 1 files changed, 111 insertions(+), 66 deletions(-) [+]
line wrap: on
line diff
--- a/docs/admin/index.rst	Wed Aug 31 12:16:08 2011 +0200
+++ b/docs/admin/index.rst	Wed Aug 31 14:48:53 2011 +0200
@@ -1,32 +1,42 @@
-====================
-Working with indexes
-====================
+=======
+Indexes
+=======
+
+General
+=======
+moin strongly relies on indexes that accelerate access to item metadata and data.
+Indexes are used internally for many operations like item lookup, history,
+iterating over items, search (also for interactive searches), etc..
+
+So, you need to configure indexing correctly first.
+
+moin will automatically update the index when items are created, updated, deleted,
+destroyed or renamed (via the storage api of moin):
+* when users change content via the user interface
+* when xml is unserialized to load items into the backend
+
+In case you have items changing without usage of the backend api or in case
+your index gets damaged or lost, you need to manually do an index build -
+or moin won't be able to work correctly.
+
 Configuration
 =============
-For correct script working you need ``index_dir`` and ``index_dir_tmp`` in
-your wiki config. They have default values and most likely you don't want to change
+Your wiki config needs ``index_dir`` and ``index_dir_tmp`` to point to different
+directories. They have default values and most likely you don't need to change
 them.
 
 But if you want, try something like::
 
-      index_dir = "/path/to/moin-2.0/wiki/index/"
-      index_dir_tmp = "/path/to/moin-2.0/wiki/tmp_build/"
+      index_dir = "/path/to/moin-2.0/wiki/index"
+      index_dir_tmp = "/path/to/moin-2.0/wiki/tmp_build"
 
 **Note:** Paths MUST BE absolute.
 
-For using one index by multiple wikis (wiki farm) you must set up ``interwikiname``
-parameter in your wiki config:
-
-Example::
 
-        interwikiname = u'MyWiki'
-
-**Note:** For correct working interwikiname must be unique for each wiki.
-
-Offline index manipulation
-==========================
-The main goal of offline index manipulation is to let wiki admin build, update, clean,
-move and monitor state of indexes.
+moin index script reference
+===========================
+You can use the ``moin index`` script to build, update, clean, move and monitor
+indexes.
 
 MoinMoin uses 2 indexes: ``latest-revs`` (index stores only current revisions)
 and ``all-revs`` (index stores all revisions).
@@ -38,32 +48,36 @@
 
 Build
 -----
-Just build fresh indexes using moin backend.
+Index all revisions of all items to the index located in ``index_dir_tmp`` (we
+use this separate location so that index building does not affect the index
+your wiki engine is currently using).
+
+If there is no index at that location yet, a new index will be built there.
+If there is already an index at that location, that index will be extended.
 
 Example::
 
     moin index --for <indexname> --action build
 
-Indexes will be built under ``index_dir_tmp`` so index building happens without
-affecting the index your wiki engine uses currently.
+**Note:** moin won't use this index until you have moved it to ``index_dir``.
+
+Move
+----
+Move indexes from ``index_dir_tmp`` to ``index_dir``.
+
+Example::
+
+    moin index --for <indexname> --action move
 
 Update
 ------
-Update indexes to reflect the current backend contents. Add new stuff, remove
-outdated stuff.
+Update the index located in ``index_dir`` to reflect the current backend
+contents. Add new stuff, remove outdated stuff.
 
 Example::
 
     moin index --for <indexname> --action update
 
-Move
-----
-Moving indexes from ``index_dir_tmp`` to ``index_dir``.
-
-Example::
-
-    moin index --for <indexname> --action move
-
 Clean
 -----
 Create empty index in ``index_dir`` for given index (previous will be erased).
@@ -74,56 +88,87 @@
 
 Show
 ----
-Showing content of index files in human readable form.
-
-**Note:** field length limited to 40 chars.
-
-**Note:** fields without attribute ``stored=True`` are not displayed.
+Show contents of the index located in ``index_dir`` in human readable form.
+This is mostly used for debugging.
 
 Example::
 
     moin index --for indexname --action show
 
-Building wiki farm
-==================
-Wiki farm allows admins create several wikis which share one index. So users
-will be able to search in one wiki and also see results from others.
+**Note:** field length limited to 40 chars.
 
-Before start you must prepair your wiki config.
+**Note:** fields without attribute ``stored=True`` are not displayed.
 
-For example, you have 3 wikis: ``Advertising``, ``Sales``, ``Engineering``
+
+Building an index for a single wiki
+===================================
+Build index at separate place, move it at right place:
+
+     moin index --for both --action build
+     moin index --for both --action move
+
+
+Building an index for a wiki farm
+=================================
+If you run a wiki farm (multiple, but related wikis), you may share the index
+between the farm wikis, so farm wiki users will be able to search in one wiki
+and also see results from the others.
+
+Before start you must prepare your wiki configs.
+
+For example, imagine some company uses 2 farm wikis: ``Sales``, ``Engineering``
 
 So, wiki configs will be looking like 
 
-wikiconfig.py for ``Advertising``::
-
-      index_dir = "/path/to/wiki/index/"
-      index_dir_tmp = "/path/to/wiki/tmp_build/"
-      interwikiname = u"Adverising"
-
-wikiconfig.py for ``Sales``::
-
-      index_dir = "/path/to/wiki/index/"
-      index_dir_tmp = "/path/to/wiki/tmp_build/"
-      interwikiname = u"Sales"
+wiki config for ``Sales``::
 
-wikiconfig.py for ``Engineering``::
-
-      index_dir = "/path/to/wiki/index/"
-      index_dir_tmp = "/path/to/wiki/tmp_build/"
-      interwikiname = u"Engineering"
+      interwikiname = u"Sales"
+      index_dir = "/path/to/wiki/index"
+      index_dir_tmp = "/path/to/wiki/index_tmp"
 
-So, after you defined configs you may start building indexes.
+wiki config for ``Engineering``::
 
-**Note:** Do not build indexes for two or more wikis in parallel, you'll damage
-it or get traceback.
+      interwikiname = u"Engineering"
+      index_dir = "/path/to/wiki/index"
+      index_dir_tmp = "/path/to/wiki/index_tmp"
 
-You must successively build index for each wiki in appropriate virtual env and then
-move indexes from ``index_dir_tmp`` to ``index_dir``::
+Now do the initial index building:
 
-     moin index --for both --action build # in Advertising virtual env
      moin index --for both --action build # in Sales virtual env
      moin index --for both --action build # in Engineering virtual env
      moin index --for both --action move # you can run it from any virtual env
 
-So, after that just run moin and try to search for something.
+Now you should have a shared index for all these wikis.
+
+**Note:** Do not build indexes for multiple wikis in parallel, this is not
+supported.
+
+Building indexes while your wiki is running
+===========================================
+If you want to build an index while your wiki is running, you have to be
+careful not to miss any changes that happen while you build the index.
+
+``moin index --action build`` is made to not interfere with your running wiki.
+So you can run this in parallel without taking your wiki offline.
+Depending on the size of your wiki, index build can take rather long - but it
+doesn't matter as you don't have to take your wiki offline for this.
+
+But: if indexing takes rather long, it can easily happen that content that was
+already put into the index is updated afterwards in the online wiki. So we need
+to do a quick index update while the wiki is offline:
+
+Offline your wiki (or at least make it read-only, so no data in it changes).
+
+``moin index --action move`` to move indexes into place.
+
+``moin index --action update`` to add anything we might have missed otherwise.
+As this is not as much as doing a full index build, this should be rather quick
+(but still: it has to look at every item in your wiki, whether it has been
+updated after the initial index build).
+
+Put your wiki back online again.
+
+**Note:** Indexing puts load onto your server, so if you like to do regular
+index rebuilds, schedule them at some time when your server is not too busy
+otherwise.
+