Vala and Reproducibility

Reproducibility, in Debian, is:

With free software, anyone can inspect the source code for malicious flaws. But Debian provide binary packages to its users. The idea of “deterministic” or “reproducible” builds is to empower anyone to verify that no flaws have been introduced during the build process by reproducing byte-for-byte identical binary packages from a given source.

Then, in order to provide reproducible binaries to Vala projects we need:

  1. Make sure you distribute generated C source code
  2. If you are a library, make sure to distribute VAPI and GIR files

This will help build process to avoid call valac in order to generate C source code, VAPI and GIR files from your Vala sources.

Because C source is distributed with a release’s tarball, any Vala project could be binary reproducible from sources.

In order to produce development packages, you should distribute VAPI and GIR files, along with .h ones. They should be included in your tarball, to avoid valac produce them.

GXml distribute all their C sources, but not GIR and VAPI files. This should be fixed next release.

GNOME Clocks distributes just Vala sources; this is why bug #772717 against Clocks, has been filed.

libgee distributes Vala sources also, but no Debian bug exists against it. May be its Vala source annotations helps, but may is a good idea to distribute C, VAPI and GIR files in future versions.

My patches to GNOME Builder, produce Makefiles to generate C sources form Vala ones. They require to be updated  in order to distribute VAPI and GIR files with your Vala project.

Should we drop Vala?

Richard Huges, on my recent post, has pointed out his interest to re-write GXml in C in order to avoid Vala dependency, quoting his words:

[…] Being honest; I’m not going to be depending on a vala library any time soon as I have to maintain all my stuff for the next decade and Vala isn’t an ideal long-term support option at all. […]

Is it Vala development a waste of time? Is Vala suitable for long term support libraries?

In GNOME, core technologies have been written in C, with high level bindings using Python and, as an intermediate solution Vala, both with strong relation to GObject Introspection.

Vala have a candy syntax, making write C API and C objects based on GLib really productive, while avoids any overhead.

While GLib have provide an Object Oriented approach because GObject, it is still C. You have to use C sentences all time even for the most common tasks. You can write really resource and speed optimized code if written in C.

GObject Introspection makes bindings to different languages available at release-day of C libraries, so for Vala ones. Vala create directly GIR required files at compilation time.

Vala generates C code. It isn’t of course, may never, perfect or reproducible. This is true and a hard issue for libraries providing interfaces, because its algorithm, I think, uses hash tables to store some parsed code representation, making impossible to generated reproducible code each time it is generated, because function position in its C form, could change producing ABI incompatible sources; this never change C behavior or security.

There are a set of fixes introduced recently to Vala to avoid interfaces issues, used by, for example libgee. This is the path for GXml to follow if it is written in Vala.

Reproducible C code generation may  require to change a lot of things internally in Vala compiler.

GLib and GObject uses macros to reduce hand written C code, Vala coders use Vala compilers because C development could be slow and error prone; and because you should care about memory management by hand too.

If Vala is not going to be maintained, because a set of core developers prefer to use C, and GNOME Builder is not going to take the time to improve its Vala support, because C is the way to go for long term support projects, then may we should stop to use Vala for libraries. Is it?

Improve Vala C code generation.

While there is macros in C to reduce burden on hand code generation, still there is room to have macros written in Vala to do the same. This is a really futuristic, wish list, sentence.

Vala syntax makes possible to write GXml, with the features it have today. Serialization framework, was possible once a GObject based API (and in Vala) was available and DOM4, was possible because we was able to copy and paste specification declarations making just few changes to Vala syntax.

Vala provides a high productive C code generator with lot of candy syntax to most common activities, like string manipulation, with automatic memory management, using secure GLib’s methods and so on.

Is it possible to improve Vala compiler to create more readable C code, suitable to switch from Vala to C?

Is it possible to embed Vala code in C or C in Vala code?

We need to explode how C generated code from Vala code, can be imported *as is* to GLib and improve over time, removing temporally variables and replace some more code to C’sh style according with the target project code standard. May this, task for GXml, is a little hard because it depends on libgee, another Vala library; but no impossible.

C limits expanded with Vala

C have lot of applications and Vala generics is pushing C limits with Generics, libgee use generics along a set of interfaces and implementations, allowing as to use different kind of collection for our objects, this powerful combination of GInteface and Vala generics, is a feature hard to reproduce (in its easy and convenient way in Vala syntax) to C API.

GObject and GInterface have a set of limits, expanded with Vala too. Now is possible to define very quickly a set of interfaces and implementations, figuring out really fast its relation ship, thanks to Vala syntax.

This advantages, in others out of the scope (and my mind), make me ask if we can embed Vala code in C or C in Vala code. Even if once a feature is mature enough in Vala code, may someone can (with C code generation optimization) import to other core libraries like GLib. Again this is one of my wish list.

Vala as a Long Term supported language

Vala syntax provides lot of advantages over hand written GObject classes, in the spirit of productivity.

GXml provides C and Vala API. Vala is more easy to use and allows to implement object oriented specifications (may be based on the idea of Java classes), but providing back a C API. Vala code is more Object Oriented syntax, than C/GObject.

Then how we can continue to contribute Vala libraries to be used by any one, and suitable to be used by any other projects even written in C?

How we can make a Vala library, like GXml, a GNOME core component with Long Term Support?

There are plenty of bugs and room to improvements to C code generation in Vala compiler, we, their users, should care about and may pay for improvements. I really want to some one add Jürg Billeter, Rico Tzschichholz or Matthias Berndt to the list of “Adopt a Hacker” list from GNOME Foundation, in order to push at least one of this ideas to improve GNOME infrastructure.

XPath on next GXml 0.14

Yanic Inizan have implemented XPath on GXml in a few hours!!! Wow!!!

It was the first requested feature for GXml after a poll in G+. Now is available upstream.

For QA reasons, we need to add Unit Tests to this XPath implementation. I really love to see tests mimic different use cases but all of XPath 3.1 specification, this will help us to know where libxml2’s implementation is and how we can improve them using GXml.

Just to comment, if some feature or function is missing, we should add an expected fail case, in order to finish correctly make check, and to leave space to change it as how it should be when this feature is considered XPath 3.1 compliant.

GXml 0.12 is here!

GXml is a Vala library providing a C and Vala API, along side Python, JavaScript and other languages bindings supported by GObject Introspection, for XML like files for read and write, like HTML, and a framework for GObject serialization to XML. It uses libxml2 as XML engine, wrapped using classes called GNode/GDocument series, and as a reader/writer of a pure GObject implementation of XML trees called TNode/TDocument.

Each implementation have its advantages, GNode have less memory usage but suffer on speed, while TNode require much more memory but is really fast. All measurements were done using Serialization/De-serialization tests for large XML files.

GXml 0.12 provides DOM4 specification implementation as main feature. This allows to deprecated the old DOM1 one. DOM4 requires API changes, but almost all Vala should compile with little modifications.

DOM4 implementation is still incomplete, but allows to you access any XML like trees, like HTML, using a GObject API.

DOM4 was possible thanks to GObject+Vala, because most of its interfaces was really easy to port using Vala syntax and GInterface. GNode/GDocument is the only classes implementing DOM4 interfaces, but other will come.

With these lasts changes, GXml API is becoming more and more stable, suitable to try new use cases.

Some one talk about to support XPath, me about to support XSD and large XML files. May you want to leave comments on what should be the next step on GXml development. Search at G+ Vala channel for a poll.

GTK+ Tester Window?

For an internal application, I’ve created a Gtk.Window derived tester class, added some widgets to show current test, status, number of fails and a Gtk.Grid to attach custom widgets. This class expose some API to set a widget to test, autoclose and some signals you can use to run some tests.


GTK+ custom widget testing is no an easy task, at least it is not integrated into GTK+, making you to figure out how to add public API or signals you can use to connect to test it. Of course, you don’t have to do the same, I just starting to create this tester window class.

My widgets, are in a library then no app is used. I’ve tested each one at a time to check it constructs without warnings, making Unit Test to pass. See my post about this warnings.

Now, when I need to see if my custom widget is showing the correct information, like in a Gtk.TreeView, is hard if you don’t API exposing it; I really don’t want to expose public API to test it, because is not the intention to allow any other to interact with each internal widgets. I’m open to see if this is the correct way to do so, but for now I really would not like to do so.

When testing a GUI widget, you should be inside a GLib loop to catch signals, may be not the only one method. Catch them allows to see if the widget is sending them; if when it is raised is showing correct information to the user (requiring to add public API to access internal widgets), and so on.

I really would like to see GNOME Builder merge my work on templates, I would like to publish this class (written in Vala), to any one to use for its custom widgets testing.

Once, I’ve finished to try to use already existing Gtk.Widget signals and API, I’ll back to suggest one feature to be added to GTK+ to make them testable.

Unit Testing GTK+ libraries: has_default GtkWidget property

For my GTK+ based library, I use Vala and Glade to create templates for custom widgets with Unit Testing. For some reason I set can_default and has_default to true, this throws a warning message:

Gtk-WARNING **: /build/gtk+3.0-z20Lwo/gtk+3.0-3.20.7/./gtk/gtkwidget.c:8550: widget not within a GtkWindow

That 8550 line is related to method gtk_widget_grab_default, when you create your new custom Gtk.Grid derived widget, before to add it to a Gtk.Window, it calls this method before it has a top level window, sending this warning.

Now, for general GTK+ applications this is not a problem; but for GTK+ libraries trying to add a kind of Unit Testing. GTest will fail your test case, when any warning is raised, then you can’t continue until you fix it.

This is to help any one with same issue and for me for future developments.

Hope to share some of my work to create a custom GTK+ Window widget for custom widget testing using  GTest; stay tuned.


Vala & Documentation: Part I

When you have developed a library using Vala programing language, or any other language, you should generate documentation about its API. You can use Valadoc to generate Documentation for Vala based libraries.

GXml, is a Vala project using valadoc command line tool and Autotools, to automatically generate its documentation. As you can see in this link, Vala documentation provides a very convenient graph about object’s hierarchy; properties and methods provided by current class; and all inherited properties and methods from base classes and interfaces it implements.

But how can I add this feature to my own Vala project? Well, you can use Autotools to call valadoc, generate your documentation and then install in the system, to be available for GNOME DevHelp to show up locally and, at the same time, in GNOME Builder integrated help facility.

While I haven’t, yet, found the way to generate C API documentation using valadoc, from my Vala API one, I’ll show you, in a second part, how you can do it, but not before I have figured out how to create a mk file you can use in your project for an out-of-the-box procedure.

Autotools Templates for GNOME Builder for GTK+ Vala applications

I’ve recently finished a makefile to make easy to create GTK+ applications as well as GTK+ libraries using Vala programing language.

While Vala allows you to use UI files generated by Glade as templates to create custom widgets, is recommended you add these ui files as resources to your application or library’s binaries, in order to build your User Interface. While Vala custom widgets are out of scope of this post, we will talk about how I’ve created a makefile to be imported into your for GTK+ resources embedding., is a file to be included in your, after you include or, depending on your project. Use the first for  GKT+ applications; the last when creating UI libraries.

Once this pre-requisites are met, you are ready to set up a GTK+ application, because, adds to your VALAFLAGS the gtk+-3.0 package, but the most important, it adds a reference to the XML file with all your resources to be embedded into your application or library binaries. This is required in order to Vala’s compiler to find all your UI files for widgets templates or simple build using GtkBuilder; also images, and any other kind of file can be added as resource, like licenses, texts, and others.

glib-compile-resources is used to generate a C source code file and its header, for all resources, to compiled with your project’s target. Once it is compiled this way, all your resources will be available using GResource or any other function using GLib resources.

For GNOME Builder, I’ll send a patch for it to add an Autotools template to create both applications and libraries for GTK+ written in Vala with resource compilation support.

May be in the near feature, this file could be splitted to include just resource generation as, in order to use it when GTK+ is not used at all, but this is another history and a good reason to write some thing about.


GNOME Builder, Unit Tests and Vala libraries

Unit testing is one of the most important features you have to write when developing a library. GLib have a test suite and is available for GLib based libraries for a while, and it is available for Vala libraries too.

Now I’ve finished a patch to add Unit Testing to Vala libraries templates in GNOME Builder. Hope it is clean and easy to use, even for other projects not using Builder as primary IDE.

This template should be available on Builder 3.22 and will help you, Vala developer, to setup your Vala libraries easy and start to write down your tests cases for bug hunting, regression-free releases and better experience for your users.

GNOME Builder, Autotools and Vala libraries

Recently blogged  about make distcheck and Vala libraries and how you can use GNOME Builder for Vala development, both are important for Vala developers expecting to increase its productivity.

As all you know Autotools is hard to learn, or at least takes some time to produce custome build rules. While Vala in Autotools have been there for a while it is buggy, making hard to get a out-of-the-box, clean and easy build system.

Now as part of my work to fix Bug #766621, I was able to produce a clean, with all passing make distcheck. I’ve added some utility makefiles to build libraries with GObject Introspection support out-of-the-box.

Just checkout my test bed project at GitHub, in order to see how simple could be.

GNOME Builder has a template system, really easy to update, and now it will have, as soon as may work on Bug #766621 is accepted to commit, out-of-the-box Vala libraries templates for your pleasure.