view docs/devel/development.rst @ 5:4641656b7102

add some short docs about testing
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Mon, 21 Feb 2011 03:23:04 +0100
parents 497a65401dcb
children d632bfadc25c
line wrap: on
line source

Project Organisation
We mainly use IRC and the wiki for communication, documentation and

IRC channels (on

* #moin-dev (core development topics)
* #moin (user support, extensions)



Issue tracker:


We use Mercurial DVCS (hg) for distributed version control.


* (main repository)
* (bb mirror for your
  convenience, simplifying forking and contributing)

If you are not using Mercurial, you can of course also just send us patches.

MoinMoin architecture
moin2 is a WSGI application and uses:

* flask as framework

  - flask-script for commandline scripts
  - flask-babel / babel / pytz / parsedatetime for i18n/l10n
  - flask-themes for theme switching
  - flask-cache as cache storage abstraction
* werkzeug for lowlevel web/http stuff, debugging, builtin server, etc.
* jinja2 for templating (theme, user interface)
* flatland for form data processing
* EmeraldTree for xml / tree processing
* blinker for signalling
* pygments for syntax highlighting
* sqlalchemy as sql database abstraction (for indexing)

  - by default using sqlite as database
* jquery javascript lib
* CKeditor - GUI editor for (x)html
* TWikiDraw, AnyWikiDraw, svgdraw drawing tools

.. todo::

   add some nice gfx

How MoinMoin works
This is just a very high level overview about how moin works, if you'ld like
to know more details, you'll have to read more docs and the code.

WSGI application creation
First, the moin Flask application is created (see ``) -
this will:

* load the configuration (app.cfg)
* register some Modules that handle different parts of the functionality

  - MoinMoin.apps.frontend - most stuff a normal user uses
  - MoinMoin.apps.admin - some stuff for admins
  - MoinMoin.apps.feed - feeds (e.g. atom)
  - MoinMoin.apps.serve - serving some configurable static 3rd party stuff
* register before/after request handlers
* initialize the cache (app.cache)
* initialize the storage (
* initialize the translation system
* initialize theme support

This app is then given to a WSGI compatible server somehow and will be called
by the server for each request for it.

Request processing
Let's look at how it shows a wiki item:

* the Flask app receives a GET request for /WikiItem
* Flask's routing rules determine that this request should be served by
* Flask calls the before request handler of this module, which:

  - sets up the user as flaskg.user (anon user or logged in user)
  - initializes dicts/groups as flaskg.dicts, flaskg.groups
  - initializes jinja2 environment (templating)
* Flask then calls the handler function `MoinMoin.apps.frontend.show_item`,

  - creates an in-memory Item

    + by fetching the item of name "WikiItem" from storage
    + it looks at the mimetype of this item (stored in metadata)
    + it creates an appropriately typed Item instance (depending on the mimetype)
  - calls Item._render_data() to determine how the rendered item looks like
    as HTML
  - renders the `show_item.html` template (and gives it the rendered item html)
  - returns the result to Flask
* Flask calls the after request handler which does some cleanup
* Flask returns an appropriate response to the server

Moin supports different storage backends (like storing directly into files /
directories, using Mercurial DVCS, using a SQL database, etc. - see

All these backends conform to same storage API definition (see
``), which is used by the higher levels (no matter what
backend is used).

There is also some related code in the storage package for:

* processing ACLs (access control lists, protecting that items get accessed
  by users that are not allowed to)
* router (a fstab like mechanism, so one can mount multiple backends at
  different places in the namespace)
* indexing (putting important metadata into a index database, so finding,
  selecting items is speedier)

DOM based transformations
But how does moin know how the HTML rendering of some item looks like?

Each Item has some mimetype (stored in metadata) - the input mimetype.
We also know what we want as output - the output mimetype.

Moin uses converters to transform the input data into the output data in
multiple steps and has a registry that knows all converters and their supported
input and output mimetypes.

For example, if the mimetype is `text/x-moin-wiki`, it'll find that the input
converter handling this is the one defined in `converter.moinwiki_in`. It then
feeds the data of this item into this converter. The converter parses this
input and creates a in-memory `dom tree` representation from it.

This dom tree is then transformed through multiple dom-to-dom converters for

* link processing
* include processing
* smileys
* macros

Finally, the dom-tree will reach the output converter, which will transform it
into the desired output format, e.g. `text/html`.

This is just one example of a supported transformation, there are quite a lot
of converters in `MoinMoin.converter` supporting different input formats,
dom-dom transformations and output formats.

Templates and Themes
Moin uses jinja2 as templating engine and Flask-Themes as a flask extension to
support multiple themes (each themes has static data, like css, and templates).

When rendering a template, the template is expanded within an environment of
values it can use. Additionally to this (general) environment, parameters can
be also given directly to the render call.


We use py.test for automated testing (it is currently automatically installed
into your virtualenv as a dependency).

Running the tests
To run the tests you first need to enter your virtualenv::

    . env/bin/activate

To run tests, enter::

    py.test  # runs all tests
    py.test -k somekeyword  # just run the tests matching somekeyword
    py.test  # just run the tests contained in

Tests output
Most is quite self-explaining, the characters mean::

    . test ran OK
    s test was skipped
    E error happened while running the test
    F test failed
    x test was expected to fail (xfail)

If something went wrong, you'll also see some traceback and stdout/stderr.

Writing tests
Writing tests with `py.test` is easy and low on overhead. You basically just
use `assert` statements.

For more information, please read on there: - but keep in
mind that we currently still use **py.test 1.3.4**.