Unittesting Gtk+ Applications

I’ve been meaning to write this for a couple of months, but I never found the time to do so. But as I promised Federico, here we go..

In this blog post I will share some of the experienced gained in unit testing a large Gtk application written in Python. Some of it only applies to Python, but most of the concepts are possible to implement in other languages.

First a bit of background. As most software projects Stoq started out with no unittests at all. That changed some 5-6 years ago when unit tests for the core business logic were added. Almost all application state of Stoq is stored in PostgreSQL, accessed via the psycopg2 database adapter and Storm, an excellent ORM written by Gustavo Niemeyer.

The current (late march 2013) code coverage of the domain classes is above 90%,
we’ve been trying to increase this gradually, but it’s a lot of work to gap the
remaining part and have never really been the focus.

The business logic, which we refer to as the domain classes, are conceptually
easy to test and has been documented elsewhere so I will not go in great detail in here.

While the domain classes are arguable the most important parts to test in the Stoq case, for instance doing payment related calculations wrong would be disastrous, we’ve always had problems of errors creeping up in level above the UI layer on top of it.

Sometimes around August last year I decided to investigate if anything could be done to
easily increase the coverage of code in the UI layer, which involves Gtk+/PyGTK.

My first approach was something like:

 dialog = PaymentEditor()
 assertEqual(dialog.value_widget.get_text(), "0")
 assertEqual(dialog.description_widget.get_text(), "")

PaymentEditor() constructs the widget tree, either manually or via a glade file,
creates an empty database domain object (Payment) and attaches it to the dialog.

Now, let’s test with real values, create a Payment and attach it to the form:

payment = Payment(value=10.0, description="New Payment")

Create another dialog showing the domain object:

dialog = PaymentEditor(payment)

And verify that the dialog has the right values set:

 assertEqual(dialog.value_widget.get_text(), "10.0")
 assertEqual(dialog.description_widget.get_text(), "New Payment")

Okay, so far so good.
This has increased the code coverage and we test opening an empty dialog,
which is used when creating a new payment and we’re testing.

There are a couple of problems with this approach though:

  • While working fine for 1 dialog and 2 fields, it’s not really scalable to huge assistents (~8 pages, 50+ widgets)
  • We’re only testing a limited subset, we want to test everything that’s visible. Labels, entries, packing, selection, columns and kiwi extensions such as validation, mandatory state, input masks.

So instead of checking each widget individually, let’s serialize the whole widget tree,
into what we call a UI-test, it looks like this:

GtkDialog(main_dialog.toplevel): title='Edit Details of "New Payment"', hidden
  GtkVBox(main_dialog._main_vbox):
    GtkVBox(main_dialog.vbox, expand=True, fill=True):
      GtkEventBox(main_dialog.main, expand=True, fill=True):
        GtkAlignment(alignment):
          GtkTable():
            ProxyLabel(description_lbl): 'Description:'
            ProxyEntry(description): 'New payment'
            ProxyLabel(value_lbl): 'Value:'
            ProxyEntry(value): '10.00', insensitive

Okay, the UI-test format has a couple of improvements:

  • basic widget hierarchy is tested (g_type_name, g_type_parent)
  • content of GtkEntry/GtkLabel are included
  • widget sensitivity (via insensitive) and visibility (via hidden)
  • packing options (expand, fill)

We also do some Python magic to include the variable names of the widgets, so it’s easier to read and understand which widget is is which, this is the value in parenthesis in the name. The dot notation means that the widget reference is stored in a sub instance of the test dialog.

This is similar to HTML page testing, where you save the rendered content of your web application to disk. We plug this into our testing infrastructure so that we have a call like:

self.check_dialog(dialog, "test.uitest")

Which serializes the widget tree to a string and compares it to the previous run, which is stored in the source code repository. If any changes are made to the widget tree, we show a diff against current and last known state, eg:

======================================================================
FAIL: stoqlib.gui.test.test_missingitemsdialog:TestMissingItemsDialog.test_confirm
----------------------------------------------------------------------
Traceback (most recent call last):
[snip]
failed:
test.uitest --- 
test.uitest +++ 
test.uitest @@ -1,9 +1,9 @@
test.uitest -dialog: ConfirmSaleMissingDialog
test.uitest +dialog: MissingItemsDialog
test.uitest GtkDialog(toplevel): title='Missing items', hidden
test.uitest GtkVBox(_main_vbox):
test.uitest GtkVBox(vbox, expand=True, fill=True):
test.uitest GtkEventBox(header):
test.uitest - GtkLabel(): '<b>The following items don&apos;t have enough stock to confirm the sale</b>'
test.uitest + GtkLabel(): '<b>The following items don&apos;t have enough stock to confirm.</b>'
test.uitest GtkEventBox(main, expand=True, fill=True):
test.uitest ObjectList(_klist):
test.uitest column: title='Product', expand

In the test above which is a real error that happened to yesterday, a GtkLabel() changed and someone forgot to update the UI-test, which is a simple rm + rerunning the test.

A real UI-test for our PaymentEditor can be found here.

In the format above, we’re also including the complete domain objects, which may contain attributes that are not shown in the interface, but nevertheless important to test.

The implementation for this, which is pretty generic and couple be used for any Gtk application currently lives inside Stoq and can be found here:

https://github.com/stoq/stoq/blob/master/stoqlib/gui/uitestutils.py

At some point I need to sit down and move it out of Stoq and put in an external library that can easily be used by other Python applications, as it currently a bit tied to kiwi/Stoq.

And the real PaymentEditor test can be found here, which shows how to use the internal API can be found here.

One important aspect of successful UI-testing involves mocking, as we need to be able to fake state to be able to test all code. But that’s for a separate posting as this is already getting a bit long.

So in summary, our total coverage before starting to do UI testing was around 35%, essentially no interfaces were tested. 8 month after first being introduced we have written 419 different UI-tests and the coverage of our currently at 78%. Remember this is a project that has 80k lines+ of Python code.

The main effect of this is that we’ve reduced the amount of QA needed before doing new releases and we have a lot more confidence that things keep working when doing large refactorization.

Posted in Blogroll, General, GNOME, olpc, PyGTK, Python | Leave a comment

Writing a mixed Gtk / Javascript application

In my last blog post I mentioned the embedding of a javascript library inside Stoq. I got a couple of requests which asked me how this was accomplished, this blog post attempts to explain some of it.

Of course we need to use the great WebKitGtk library. Unfortunately we cannot use the introspection based bindings as this needs to work on Gtk+ 2.18 and PyGTK 2.17 which were shipped in the last Ubuntu LTS release.

WebView will do all the html/css/js parts. It’s almost as simple as a normal GtkTextView, add it to a scrolled window, load the content and off you go.

The first challange comes when you want to open http:// links in your normal browser, instead of handling them in your webkit. To do that you need to listen the navigation-policy-decision-requested signal ignore certain requests. You don’t actually need to use http protocols, you can invent any url which is parsable.

Next problem is AJAX, to write a proper asynchronous widget you don’t want to reload the whole page when something changes. Since we cannot implement our own protocols in the old libsoup bindings shipped for PyGTK we need to run our own http server. That is good for other reasons as well, we can do heavy IO such as database queries in there without actually blocking the user interaction.

When we need to execute scripting in gtk we just call web_view_execute_script() which will just execute a piece of javascript. For instance,

view.execute_script(“document.title = $(‘fc-header-title’).text()” is a actual line in Stoq, it sets the window title based on calendar header title from the dom.

Going the other direction is a bit uglier, the only way of communcation I found out was opening new urls, so I implemented an application specific domain which opens a dialog or some other action within the gtk application.

I know that some of these tricks are already outdated, in newer webkitgtk versions you should write your own libsoup handlers, use the gobject dom bindings for communication, but I didn’t have these options when writing this post

TL;DR

  • Listen to the ::navigation-policy-decision-requested to implement your own uri handling if you can’t do it via libsoup.
  • Run a separate daemon process which will serve as an internal webserver, so AJAX calls work and won’t block on IO.
  • Use web_view_execute_script() for Gtk->Javascript communication
  • Use window.location = “customprotocol://”  for Javascript ->Gtk communication
  • Make the javascript parts work in a normal browser so the normal developer tools can be used
  • If possible, use a newer version of WebKitGtk and avoid all of this.

Xan and the other webkit hackers will probably look at me in disgust for telling you how to do all of these dirty hacks 😉

Posted in Blogroll, General, GNOME, olpc, PyGTK, Python | 5 Comments

Stoq 1.2

We released Stoq 1.2 last week, this release features quite a bit of features:

Calendar application

It’s now possible to list payments, purchase orders and client calls in a graphical view:

 

It might look familiar, it uses the fantastic javascript library fullcalendar. We really wanted to use a normal GtkWidget for the calendar but it would have been a lot more work to rip out half of evolution. If there are any other options that can match fullcalendars functionallity there we’d be open to switching as embedding WebKit, jQuery and fullcalendar in a Gtk+ application is not ideal.

Configurable keyboard shortcuts

This is something that has been requested many times over the years. It makes it easier to remap the keyboard bindings use often to other keys, such as the function keys. There’s still an open task to redo all the existing keybindings that aren’t uniform enough.

Configurable form fields

Some companies does not use all the form fields (fax anyone?) that we show per default and Stoq know has a configuration interface where you can make fields non-mandatory and even hide them if you don’t wish to see them. Perfect for the first steps of localization.

New manual

One of our interns rewrote old docbook manual to mallard, and it looks beatiful and is now well integrated in the application. You can find the online version here. It involved removing a lot of screenshots and text. It’ll be easier to update the manual in the future if there aren’t any screenshots. He also fixed the interface, there are now various help buttons in the application that goes to a help section describing that part.

Localization support

It’s now possible to configure some of the fields that are specific to each region/country. The only thing that made it into this release was company identification number (Brazil: CNPJ, Sweden: Organisationnr, US: Employer Identification Number). But person identification number and list of states has landed in the code repository since the release. We still need someone to step up and start doing the actual localization for this, be the hero of the day and download Stoq and start localizing it!

Boleto Bancário (Bank invoice)

Brazilian banks supports a kind of invoice with a barcodes/numbers, called boleto bancário. It’s semi-standardadized, most of the data is similar, but you need to special case each bank that should be supported. There are two kinds, with and without cobrança (for eventually sending to a collection agency). There are a couple of 100 active banks and about 15 major ones. Stoq currently supports 7: Banco do Brasil, Banco Real, Banco Santander, Banco Bradesco, Caixa Econômica, Banrisul, Banco Itaú. All without cobrança though, support for that will come in a future release.

Call for volunteers

Stoq has initially been targeting the Brazilian market, since that what’s close to the current development team. But there is now longer an excuse for not trying to use it. We can barely handle the legal part of Brazil and we’d need volunteer help to make it possible to use in other countries. We’re very proud of the application so we wouldn’t want to stop you just because you live outside of Brazil!

So, why don’t you grab the code and get started, it’s all python (and a tiny bit of javascript) and shouldn’t be hard to get started.

Don’t be discouraged by the web site and manual is only in Portuguese, we use gettext and rosetta and the code is modular and easy to understand.

We’ll need a lot of work to support localization in different countries such as: company/person formats, states, taxes and other things we don’t know about yet, let us know and we’ll try to find a solution.

Just send me a mail or come in on our new shiny web chat: http://chat.stoq.com.br/ (aka #stoq on freenode)

Posted in Blogroll, General, GNOME, olpc, PyGTK, Python | 2 Comments

Launching Stoq 1.1

We’re launching Stoq 1.1 today, which is a POS/CRM/accounting/industrial application mainly for the Brazilian market. In short the kind of software you’d use in a supermarket, bakery, cloth shop, manufacturing company etc.

It’s written in Gtk/Python/PostgreSQL and is available for Linux/Windows. Free software (GPL), stable, fast and beautiful!

New for this release is the reorganization of the applications to always fit into one window (conceptually similar to evolution/outlook) and a new application selector which should make it easier to switch between different parts of the app.

It’s fully translated and can be used outside of Brazil, even though a few small details are oriented around the fiscal requirements set by the brazilian government.

There’s a livecd available on Ubuntu, so you can just download it and try it out.

For more information see http://www.stoq.com.br/

Posted in Blogroll, General, GNOME, olpc, PyGTK, Python | 1 Comment

Interview in Portuguese

A couple of weeks ago I did another first – a recorded interview in Portuguese, thanks to the great host Og Maciel.

I had a great time and we talked about my involvement in GNOMEStoq together with a bunch of other random topics, check it out at:

http://www.castalio.info/johan-dahlin-stoq-gestor-comercial-livre/

Posted in Blogroll, General, GNOME, olpc, PyGTK, Python | Leave a comment

Using LLVM to speed up function invocation in a dynamic language binding

With the recent release of PyGObject I decided to take a look into something I’ve been meaning to do for some time, using the excellent LLVM library to make things go faster.

But first some brief history. Back in the old PyGTK and pygtk-codegen days all functions were created at compile time generating all the Python<->C wrappers that you would ever need. This is problematic for various reasons:

  • each binding needs to be generated before using it and thus you need a Python extension module for each library you use.
  • the code generator generated straight-forward wrappers. Not a problem for a small library, but Gtk has almost 200 classes and more than 3000 functions/methods which all need to be registered when you import gtk, which takes time and memory.
GObject-introspection and PyGI solves both of these problems. Typelibs are created in the upstream library and the classes and functions wrappers are created in runtime when you use them. One important thing changed though. Invokation of the native functions are no longer done through generated code, we’re using libffi to do function invokation completely in runtime. This works pretty well but it’s much slower than the old generated functions.
This is where LLVM enters the picture. LLVM is a compiler infrastructure framework. It’s basically a full C/C++ compiler/assembler/linker split into reusable libraries. I decided to take a look into using LLVM to optimize the function invokation by generate native code for the functions lazily using the the LLVM JIT.
I can now announce that after experimenting with it I’ve been able to get it to a point where it’s usable enough to run benchmarks on. That doesn’t mean it’s feature complete, as only a few types such as double,int32,int64,object, so it’s far from supporting a complete application. Anyway, let me present some numbers. First function,
The first two ares an extreme examples as the functions does very little. But it’s great as it will *only* exercise the time it takes to convert an argument to/from native types.
The last function is not speed up considerably because the time spend most of its time inside the function itself, instantiating GObjects is far more computational expensive than converting Python wrappers back and forth.
This work in progress can be found in the llvm branch of pygobject
Posted in Blogroll, General, GNOME, olpc, PyGTK, Python | 7 Comments

Bridging the development gap between desktop and web

One of the reasons JavaScript was chosen as the language for GNOME Shell is as Owen mentions: “… a lot of people are familiar with it from the web.”. It’s not a new idea but certainly an exciting one. Who wouldn’t want to be able to run their desktop application on the web or their web application in a desktop environment?

A fairly common complaint about Gjs is that it’s not possible to use cairo from it. Cairo bindings were recently committed to Gjs, but I didn’t quite find the time to write a couple of nice applications demonstrating how to use it. But instead of writing my own blingy cairo application I thought, wouldn’t it be nicer if I could just take a canvas demo from the web and with minimal modifications run it on top of gjs with cairo?

The missing piece to that goal is a HTML5 Canvas & CanvasRenderingContext2D API on top of the cairo binding. After a couple of bug fixes in gjs I got it to a point where I could run a slightly modified version of a simple clock example in gjs:

The screenshot aboves shows the same code running in the browser to the left, in HTML. To the right is the same code running on top of Gtk+, before you ask, the source code can be found here on my github page. If you want to run this you’ll need patches from these two bugs applied first.

There are a couple of other APIs that would be nice to have in Gjs such as DOM, Storage, WebGL. I didn’t try to implement these, but let’s hope someone does so the GNOME developer environment can provide familiar apis for the huge pool of web developers out there.

Posted in General | 7 Comments

News from the introspection world

There has been a number of recent developments in the world of introspection, let me summarize some of them:

First of all, the litl got uncovered and released a couple of weeks ago. The whole litl software platform is heavily dependent on the introspection parts and on the gjs, the spidermonkey javascript bindings. It’s been a great experience of easily being able to move code between javascript and C as we see fit. Adding a new method to a GObject class in C is just a make a away from making it available to the javascript bindings. Lucas and Scott have written more about the technical parts of the litl.

Gjs has recently gained callback support, which has been kind of a personal pet peeve for me. Making sure that functions and methods with callbacks are possible to call from a language binding has always been difficult. GClosure sort of solves that problem, unfortunately very few libraries actually use them. Most of the tricky work of landing callback support in Gjs was done by Maxim Ermilov, rock on!

The Python bindings are also moving along pretty quickly. Tomeu Vizoso and Simon Van Der Linden has been working aggressively on making them solid. The rumor tells me that Tomeu has the whole OLPC Sugar interface running on top the new bindings, with significant speed and memory improvments!

Elliot Smith at Intel blogged about his experiences on using Gjs on Moblin developing a clutter application. Zach Goldberg writes about his experiences on adding callback support to the python bindings.

More and more libraries are adding introspection support, the latest ones are libgda and moblin toolkit.

Posted in Blogroll, General, GNOME, olpc, PyGTK, Python | 6 Comments

litled

Today is my first day at litl. I’m joining a team which should be familiar to GNOME:rs in general: Johan Bilien, Lucas Rocha, Havoc Pennington and Tommi Komulainen. Exciting times!

Posted in General, GNOME | 6 Comments

Introspection hackfest at the Boston Summit

We’re arranging an introspection hackfest at the Boston Summit! Thanks to the nice lads at the foundation we can have most of the participants flown over. Colin Walters, Jürg Billeter, Philip Van Hoof, I and hopefully Havoc Pennington is going to be there.

If you’re interested in language bindings or other uses of the introspection data, come by and help us out.

Oh, I almost forgot to say hi to Planet Python. Andrew was kind enough to add me there. My name is Johan Dahlin and I’m mostly working on Python related to GNOME, especially, Python bindings for GObject and Gtk., and Kiwi + Stoq of course. The introspection project is intended to make it easier to write language bindings for the Gnome stack, not only Python ones.

Posted in Blogroll, General, GNOME, olpc, PyGTK, Python | Leave a comment