The XML GUI definition files used in Rapicorn and also in Beast (described briefly in an earlier blog post) supported a simple $(function,arguments...) evaluation syntax, similar to GNU Make. I’ve never been very happy with this syntax, but it was fairly easy to implement at the start and followed naturally from early $VARIABLE expansion features. At some point last year I considered various alternative syntax variants and discussed the ideas with Stefan Westerfeld. Over the course of the last two months, I finally got around to implement them.

I’ve never grown familiar with reverse polish notation, and although Guile is the canonical scripting language for Beast, I’ve always found myself very inefficient with expressing my thoughts in lisp expressions. So the new syntax definitely had to support infix expressions – despite the more complex parsing logic required to parse them. Bison already ships with an example calculator that parses infix expressions, so that’s a quick start as far as the syntax rules go. Integration is quite a different story though, more on that later.

Since in Rapicorn the expressions are used to compute widget property values, they are likely to be executed very often, i.e. each time a widget is created from an XML file description. Consequently, I wanted to use a pre-parsed AST to carry out the evaluation and avoid mixing evaluation logic with parser logic, which would have forced reparsing expressions upon each evaluation. At first I quickly threw together some C++ classes for the arithmetic operators and used those as nodes for the AST, similar to:

  class ASTNodeNot : virtual public ASTNode {
    ASTNode &m_operand;
  public:
    explicit ASTNodeNot (ASTNode &operand) :
      m_operand (a)
    {}
    virtual Value
    eval (Env *env) const
    {
      Value a = m_operand.eval();
      return Value (!a.asbool());
    }
  };

The supported syntax is quickly summarized:

  Operators: ( + - * / ** < > <= >= == != or and not )
  Functions: name ( args... )
  Inputs:    FloatingPoint 'SingleQuotedString' "DoubleQuotedString"

In this notation, FloatingPoint includes hexadecimal numbers and of course integers and the quoted strings support C style escape sequences like octal numbers, ‘\n‘, ‘\t‘ and so on. The functions can be implemented by the parser API user, but a good set of standard arithmetic functions is already supported like ceil(), floor(), min(), max(), log(), etc. So it’s a very basic parser, but covers the vast majority of expressions needed to position widgets or configure packing properties.

One thing that turned out to be tricky is the binary operator semantics for strings. At the very least, I wanted support for "string" + "string" and "string" == "string". Since both operators are supported for numbers as well, the exact behavior of "string" + FloatingPoint and "string" == FloatingPoint had to be defined. I managed to find a few programming language precedents here in Perl, Python, and ECMAScript (Javascript). They of course all handle the cases differently. In the end I settled with ECMAScript semantics:

  Value1 == Value2  # does string comparisons if both values are strings
  Value1 + Value2   # does string conversion if either value is a string

Unit testing for the parser turned out to be particularly easy to implement. All that’s needed is a small utility that reads expressions and prints/verifies the evaluation results. Throwing in some additional shell code allowed a normal text file to drive unit testing. It simply contains expressions and expected results on alternating lines. Btw, libreadline can be really handy in cases like this. Using it takes a mere 5-10 lines of additional code to support a nice interactive command line interface including history for the evaluator test shell.

After some initial testing, the C++ AST node classes seemed like an awful lot of pointers, fragmentation and VTable calls for a supposedly straight forward expression evaluation. Also, adding the missing destruction semantics would have significantly increased the existing class logic. So I tried to come up with a leaner and foremost flat memory presentation. In the end, I settled with a single array that grows in 4 byte (integer) steps, embeds strings literally (padded to 4 byte alignment) and uses array offsets instead of pointers for references. A single multiplication operator is encoded with 3 integers that way: MUL_opcode factor1_index factor2_index. That’s essentially 12 bytes per binary operator, still significantly more than the parser input, but also significantly smaller than allocating an equivalent C++ class. Evaluation of the expression stored in the array doesn’t need any VTable calls, and a single straight forward evaluation function can be used, that implements the different operators as switch statement cases. Also release semantics are persuasively trivial for a single consecutive array.

What’s left was to figure a way to embed expression evaluation in XML values or text blocks. Previously, $(expression) was substituted everywhere, but with the new syntax, variables (or rather constants defined within the Rapicorn core or via the ‘<arg:ArgName default="5"/>‘ syntax supported by Rapicorn XML files) didn’t use a $-prefix anymore. So sticking with $() seemed to make little sense. As it’s implemented now, backticks are used to cause expression evaluation, with the empty expression evaluating to a single backtick:

  XML Value/Text         Parser Result
    Foo  5 + 5      ->     Foo  5 + 5
    Foo `5 + 5`     ->     Foo 10
    ``Foo``         ->     `Foo`

We will see how useful the current expression style turns out to be. I don’t consider every implementation bit outlined here solidly engraved into stone yet. So as always, I’m open to constructive feedback.

As forewarned, I have a few more words on integrating Flex and Bison with each other and into one’s own library. First, Flex and Bison turned out not to be exactly simple to configure, especially if none of the generated symbols should be exported from a library or clash with a possible second parser linked into the same library or program. Also some fiddling is required to pass on proper line count information from the lexer to the parser, getting character counts as well is non-trivial but lucky wasn’t strictly needed for Rapicorn expressions. The simplest setup I managed to come up with works as follows:

  sinfex.hh     # public API
  sinfeximpl.hh # internal structure definitions
  sinfex.cc     # evaluator implementation
  sinfex.l      # scanner rules for Flex
  sinfex.y      # parser rules for Bison
  sinfex.lgen   # generated by Flex
  sinfex.ygen   # generated by Bison

The only compiled file in this list is sinfex.cc which includes sinfex.lgen and sinfex.ygen as part of an anonymous C++ namespace. A linker script ldscript.map used when finally linking the library prevents anonymous symbols from being exported. The anonymous namespacing of everything declared in sinfex.lgen and sinfex.ygen is what prevents clashes with a possible second parser linked into the library. This isn’t as elegant as i was hoping for, but at least effective in a practical sense. There unfortunately is no way to configure Flex or Bison to generate only static functions and variables. And yes, I have also looked into variants like flex++, bison++, bisonc++, byacc, etc, but they usually show much of the same problems and also tend to make matters worse by generating more files or more complex code.

Rapicorn 8.4.0 has just been released to the world: Rapicorn v8.4.0 Announcement

Lots of things have happened since the last snapshot over a year ago, some of which kept me from making releases or snapshots earlier. ;-)
Others actually took place in the code base as interesting developments, summarized below.

There’s quite some more stuff in my development queue, but at some point one just has to draw a line and throw out what’s vaguely ready so far, so this is it for today:

  Overview of changes in Rapicorn 8.4.0:

  * Changed versioning scheme to YEAR.WEEK.REVISION.
  * License update to GNU LGPL 2.1.
  * Added a publically installed tool: rapidrun
  * Support println() and close() commands in GUI files.
  * Introduce simple Application and Window object APIs.
  * Merged libbirnet into Rapicorn as librapicorncore.
  * Implemented expose region merging/comprssion.
  * Reiimplemented rectangle gradient shader.
  * Switched to autogenerated ChangeLogs.
  * Improved feedback on parser errors.
  * Fixed Gtk+ version checks.
  * Added PNG saving support.
  * Removed PERL build dependency.
  * Rewrote asyncronous main loops.
  * Many improvements to text editing areas.
  * Speed up blitting logic for local displays.
  * Added SIMD optimized rendering functions for MMX CPUs.
  * Fixed some reference counting issues and child removal.
  * Improved vertical text ellipsization to line granularity.
  * Removed error prone default values from property mechanism.
  * Install tutorial under ${prefix}/doc/rapicornXXXX/tutorial/.
  * Misc compiler and threading fixes, depend on g++-3.4.6.
  * Lots of bug fixes, cleanups and improved test coverage.

-> rapicorn.org <-

This should help to clear up some of the motivations behind the Rapicorn project, in particular how it relates to Gtk+ and why it aims at implementing features that are to some extend already covered by Gtk+ or other projects like GnomeCanvas.
Here is a small excerpt to wetten the appetite:

	These days Gtk+ is [...] a very successful toolkit.
	However, maintenance and continuous development of a project at this scope
	and scale come at significant costs in terms of development flexibility
	and compatibility. It is not anymore a suitable place for evolution of new
	experimental GUI technologies and quick paradigm shifts. So radically new
	toolkit approaches or design ideas now need to be explored elsewhere and
	only if successful and applicable can reflect back onto Gtk+ development.
	-
	Rapicorn explores some approaches which are simply different from
	established Gtk+ paradigms.

Rapicorn 0.1.2:
* added Image support for PNG images.
* added HSlider, VSlider, Arrow, dot-grid.
* added ScrollArea.
* added Adjustments.
* added command system allowing on-click properties.
* added reset logic to recover from stale event situations.
* added requisition tuning to allow iterative size allocations.
* added event grabbing.
* added MainLoop.
* added Thread-per-Window paradigm.
* added Viewport abstraction of drawing backends.
* added color schemes.
* added simple function Evaluator for XML properties.
* added Markup to labels.
* added focus handling for Items.
* added TextEditor prototype.
* added Region to handle rectangle arrays.
* added partial screen updates.
* removed libcairo/libpixman dependency.
* overhauled Table shrinking logic.
* upgraded Birnet library.

Known Issues:
+ instabilities/hangs upon main loop exit.
+ possible build issues with g++ versions < 3.4.6.

Rapicorn 0.1.2:
* added Image support for PNG images.
* added HSlider, VSlider, Arrow, dot-grid.
* added ScrollArea.
* added Adjustments.
* added command system allowing on-click properties.
* added reset logic to recover from stale event situations.
* added requisition tuning to allow iterative size allocations.
* added event grabbing.
* added MainLoop.
* added Thread-per-Window paradigm.
* added Viewport abstraction of drawing backends.
* added color schemes.
* added simple function Evaluator for XML properties.
* added Markup to labels.
* added focus handling for Items.
* added TextEditor prototype.
* added Region to handle rectangle arrays.
* added partial screen updates.
* removed libcairo/libpixman dependency.
* overhauled Table shrinking logic.
* upgraded Birnet library.

Known Issues:
+ instabilities upon main loop exit.
+ possible build issues with g++ versions other than 3.4.

In order to get a scrollable area implemented, i first started out with the h/v slider implementations. That in turn required sorting out of the basic threading and main loop models (Rapicorn supports thread-per-window mode) to get timers going. I then ended up implementing 3 different gadgets to make up the sliders: SliderArea, SliderTrough and SliderSkid. I managed to keep the code generic enough with regards to packing abilities and movement to support horizontal and vertical scales as well as scrollbars with these 3 gadgets.

For the curious, an XML excerpt that packs together the necessary frames, buttons, boxes, ambience (shading) and functional gadgets for a vslider is here: Rapicorn vslider definition (23 lines).

Since Rapicorn currently has to support Gtk+/Gdk >= 2.6.x as rendering backend, scrolling had to be implemented with gdk_draw_drawable() + gdk_event_get_graphics_expose(). Once this dependency can be raised, we’ll use gdk_window_move_region() however. Tonight, i managed to complete the coordinate offset handling and got event processing going inside the scroll area, so here’s a button toggled inside a scroll area:

A couple other things also got fixed, like the Appearance/ColorScheme handling. E.g. here’s a blue test window, where the color scheme is derived from a set of just 3 base colors:

I’ll probably put up another tarball soon, after some polishing of the build setup and the window/main loop destruction phases.

 Text &text = container.interface<Text>();

This can be extended to allow passing in widget ids, for instance to tell apart the buttons of a dialog:

 Button &button = file_selector.interface<Button> ("cancel-button");

And to access the text of the cancel button:

 Label &label = file_selector.interface<Label> ("cancel-button");

The core idea is to build larger widgets out of micro widgets (very simple things like frame, label, table, image, rectangle shader, etc.) by writing concise xml statements. This may look like so:

		<button hexpand="1" left-attach="50" bottom-attach="59" >
		  <label _text="Expand Button" />
		</button>

This would be a button, packed into a table at 50-51 horizontally and 59-60 vertically. Also, the button is expanded horizontally within the table container and contains a label with translated text. These xml style widget definitions were originally developed for Gtk+ code and can be found as “radgets” in recent beast releases. The notation is powerful and conscise enough that nearly all beast dialogs could be defined in a single xml file.
Another major design aspect is strict seperation of model, view and controller logic, down to such simple things like toggle buttons. Most popular toolkits out there already allow seperation between model and view, but fail short on seperating the controller logic and making it pluggable (customizable). Most often this causes code more complicated than neccessary, where the view and controller logics are closely tied up, and result in hacks or code duplication in third party products to change controller behaviour as desired. Turning controller functionality into its own object instance which may be customized or replaced by the toolkit user, will help to reduce third party hacks and increase structure of the toolkit code.
Last but not least, an often heared request for beast was changing its outlook, so it looks more like “cool” music applications found under windows: (AU), (RE), (CU). This will involve working more with images, font rendering, alpha blending and dithering for standard GUI components than is intended with stock Gtk+, so this kind of featurism is also supposed to end up in rapicorn instead.
That being said, rapicorn hasn’t left initial design phase yet and much of what is there or will be there soon may not survive future releases because we’re still testing out new ideas and concepts. Semi frequent releases however will be made available at the rapicorn ftp directory, mostly in response to someone asking me to put up a new tarball.

	inline uint32 quick_rand32 (void)
	{
	  static uint32 accu = 2147483563;
	  accu = 1664525 * accu + 1013904223;
	  return accu;
	}

This has been the major speed-up-knob, other aspects were as confusing as profiling often gets, e.g. -ffast-math actually slowed down the rectangle shader by 13%.