Mercurial > moin > 2.0
changeset 811:d7b0946924f9 pytest2
improve indexing admin docs
author | Thomas Waldmann <tw AT waldmann-edv DOT de> |
---|---|
date | Wed, 31 Aug 2011 14:48:53 +0200 |
parents | 449c081c2a50 |
children | 39fc3b341002 |
files | docs/admin/index.rst |
diffstat | 1 files changed, 111 insertions(+), 66 deletions(-) [+] |
line wrap: on
line diff
--- a/docs/admin/index.rst Wed Aug 31 12:16:08 2011 +0200 +++ b/docs/admin/index.rst Wed Aug 31 14:48:53 2011 +0200 @@ -1,32 +1,42 @@ -==================== -Working with indexes -==================== +======= +Indexes +======= + +General +======= +moin strongly relies on indexes that accelerate access to item metadata and data. +Indexes are used internally for many operations like item lookup, history, +iterating over items, search (also for interactive searches), etc.. + +So, you need to configure indexing correctly first. + +moin will automatically update the index when items are created, updated, deleted, +destroyed or renamed (via the storage api of moin): +* when users change content via the user interface +* when xml is unserialized to load items into the backend + +In case you have items changing without usage of the backend api or in case +your index gets damaged or lost, you need to manually do an index build - +or moin won't be able to work correctly. + Configuration ============= -For correct script working you need ``index_dir`` and ``index_dir_tmp`` in -your wiki config. They have default values and most likely you don't want to change +Your wiki config needs ``index_dir`` and ``index_dir_tmp`` to point to different +directories. They have default values and most likely you don't need to change them. But if you want, try something like:: - index_dir = "/path/to/moin-2.0/wiki/index/" - index_dir_tmp = "/path/to/moin-2.0/wiki/tmp_build/" + index_dir = "/path/to/moin-2.0/wiki/index" + index_dir_tmp = "/path/to/moin-2.0/wiki/tmp_build" **Note:** Paths MUST BE absolute. -For using one index by multiple wikis (wiki farm) you must set up ``interwikiname`` -parameter in your wiki config: - -Example:: - interwikiname = u'MyWiki' - -**Note:** For correct working interwikiname must be unique for each wiki. - -Offline index manipulation -========================== -The main goal of offline index manipulation is to let wiki admin build, update, clean, -move and monitor state of indexes. +moin index script reference +=========================== +You can use the ``moin index`` script to build, update, clean, move and monitor +indexes. MoinMoin uses 2 indexes: ``latest-revs`` (index stores only current revisions) and ``all-revs`` (index stores all revisions). @@ -38,32 +48,36 @@ Build ----- -Just build fresh indexes using moin backend. +Index all revisions of all items to the index located in ``index_dir_tmp`` (we +use this separate location so that index building does not affect the index +your wiki engine is currently using). + +If there is no index at that location yet, a new index will be built there. +If there is already an index at that location, that index will be extended. Example:: moin index --for <indexname> --action build -Indexes will be built under ``index_dir_tmp`` so index building happens without -affecting the index your wiki engine uses currently. +**Note:** moin won't use this index until you have moved it to ``index_dir``. + +Move +---- +Move indexes from ``index_dir_tmp`` to ``index_dir``. + +Example:: + + moin index --for <indexname> --action move Update ------ -Update indexes to reflect the current backend contents. Add new stuff, remove -outdated stuff. +Update the index located in ``index_dir`` to reflect the current backend +contents. Add new stuff, remove outdated stuff. Example:: moin index --for <indexname> --action update -Move ----- -Moving indexes from ``index_dir_tmp`` to ``index_dir``. - -Example:: - - moin index --for <indexname> --action move - Clean ----- Create empty index in ``index_dir`` for given index (previous will be erased). @@ -74,56 +88,87 @@ Show ---- -Showing content of index files in human readable form. - -**Note:** field length limited to 40 chars. - -**Note:** fields without attribute ``stored=True`` are not displayed. +Show contents of the index located in ``index_dir`` in human readable form. +This is mostly used for debugging. Example:: moin index --for indexname --action show -Building wiki farm -================== -Wiki farm allows admins create several wikis which share one index. So users -will be able to search in one wiki and also see results from others. +**Note:** field length limited to 40 chars. -Before start you must prepair your wiki config. +**Note:** fields without attribute ``stored=True`` are not displayed. -For example, you have 3 wikis: ``Advertising``, ``Sales``, ``Engineering`` + +Building an index for a single wiki +=================================== +Build index at separate place, move it at right place: + + moin index --for both --action build + moin index --for both --action move + + +Building an index for a wiki farm +================================= +If you run a wiki farm (multiple, but related wikis), you may share the index +between the farm wikis, so farm wiki users will be able to search in one wiki +and also see results from the others. + +Before start you must prepare your wiki configs. + +For example, imagine some company uses 2 farm wikis: ``Sales``, ``Engineering`` So, wiki configs will be looking like -wikiconfig.py for ``Advertising``:: - - index_dir = "/path/to/wiki/index/" - index_dir_tmp = "/path/to/wiki/tmp_build/" - interwikiname = u"Adverising" - -wikiconfig.py for ``Sales``:: - - index_dir = "/path/to/wiki/index/" - index_dir_tmp = "/path/to/wiki/tmp_build/" - interwikiname = u"Sales" +wiki config for ``Sales``:: -wikiconfig.py for ``Engineering``:: - - index_dir = "/path/to/wiki/index/" - index_dir_tmp = "/path/to/wiki/tmp_build/" - interwikiname = u"Engineering" + interwikiname = u"Sales" + index_dir = "/path/to/wiki/index" + index_dir_tmp = "/path/to/wiki/index_tmp" -So, after you defined configs you may start building indexes. +wiki config for ``Engineering``:: -**Note:** Do not build indexes for two or more wikis in parallel, you'll damage -it or get traceback. + interwikiname = u"Engineering" + index_dir = "/path/to/wiki/index" + index_dir_tmp = "/path/to/wiki/index_tmp" -You must successively build index for each wiki in appropriate virtual env and then -move indexes from ``index_dir_tmp`` to ``index_dir``:: +Now do the initial index building: - moin index --for both --action build # in Advertising virtual env moin index --for both --action build # in Sales virtual env moin index --for both --action build # in Engineering virtual env moin index --for both --action move # you can run it from any virtual env -So, after that just run moin and try to search for something. +Now you should have a shared index for all these wikis. + +**Note:** Do not build indexes for multiple wikis in parallel, this is not +supported. + +Building indexes while your wiki is running +=========================================== +If you want to build an index while your wiki is running, you have to be +careful not to miss any changes that happen while you build the index. + +``moin index --action build`` is made to not interfere with your running wiki. +So you can run this in parallel without taking your wiki offline. +Depending on the size of your wiki, index build can take rather long - but it +doesn't matter as you don't have to take your wiki offline for this. + +But: if indexing takes rather long, it can easily happen that content that was +already put into the index is updated afterwards in the online wiki. So we need +to do a quick index update while the wiki is offline: + +Offline your wiki (or at least make it read-only, so no data in it changes). + +``moin index --action move`` to move indexes into place. + +``moin index --action update`` to add anything we might have missed otherwise. +As this is not as much as doing a full index build, this should be rather quick +(but still: it has to look at every item in your wiki, whether it has been +updated after the initial index build). + +Put your wiki back online again. + +**Note:** Indexing puts load onto your server, so if you like to do regular +index rebuilds, schedule them at some time when your server is not too busy +otherwise. +