17.05.2006 The Beast Documentation Quest (3)

See: http://blogs.testbit.eu/timj/2006/05/16/17052006-the-beast-documentation-quest-3/ (page moved)

Faced with the documentation system requirements outlined in the last episode (The Beast Documentation Quest (2)) it took a while to figure that there really was no single documentation tool fulfilling all the needs outlined. So writing a new documentation tool for the Beast project looked like the only viable solution. I wasn’t ready to start out rolling a new full fledged C++ parser though…
Fortunately, Doxygen includes an XML generation back-end, that will dump all the C/C++ structure after the parsing stage, including comments, into a set of XML files. Unfortunately it unconditionally preprocesses the source code comments. So it required some trickery to circumvent complaints about unknown or invalidly used Doxygen macros when combined with a newly developed macro processor…
To get the macro processor going, i wrote a recursive descend parser for “@macro{arguments} more text\n“-style constructs in Python. Added up a syntax facility for new macro definitions, an algorithm to detect and generate sections plus a rudimentary HTML generation back-end, and voila – a new documentation generator was born: Doxer. That was around last summer.

In response to my last blog entry on this topic, one commenter wrote:

I think that starting another documentation project would be totally useless.

I don’t think i can agree here. For any software solvable problem you have, you can only write your own tool if you don’t find an existing one that covers your needs or could be extended to do so. Admittedly, i didn’t know about Synopsis back then, and i intend to investigate more of its innards at some point to figure whether it can be integrated with Doxer nicely. Until then, Beast does again have a working documentation tool, re-enabling us to keep documenting the source code and updating the website.
At this point, the CVS version of Beast has been fully migrated to generate and display HTML runtime documentation, using a browser according to the users preferences. Also all source code documentation was converted and the website is fully generated from Doxer. All of this is driven by circa 6k lines of code, made possible only by the expressiveness of Python and by using Doxygen as a source code parsing backend.