Large number of XML Nodes and GXml performance

GXml performance has been improved since initial releases.

First implementation parse all to libxml2 tree and then to a GObject set of classes, in order to provide GObject Serialization framework.

Over time GXmlGom was added as a set of classes avoiding to use libxml2 tree improving both memory and performance on Serialization.

GXml has been used in many applications like parse Electrical Substation Configuration Language files by librescl.org; to Mexican Tax Authority XML invoices format, among others.

QRSVG Performance

For my private projects, I need to create QR of size 61×61 = 3721 squares. This means at least 2700 XML nodes. This is a large number of nodes and because QRSVG depends on GSVG and it depends on GXml, all them depend on GXml’s implementation for performance.

Initial measurements suggest that, at no surprise, using a simple array of objects takes up to 0.5 seconds to add just a node, as maximum time measured.

So GXml’s implementation should be improved for large number of nodes. Now it uses Gee.ArrayList, is clean and easy to wrap a node list implementing W3C DOM4 API. But now I’m considering to use Gee.TreeMap, because it is designed for large collection of objects, from its documentation:

This implementation is especially well designed for large quantity of data. The (balanced) tree implementation insure that the set and get methods are in logarithmic complexity.

The problem is its Map interface, where I need to implement a Gee.BidirList interface over it, in order to ensure fit in W3C DOM4 API and get performance boost.

Lets see how evolves this. Any suggestion?

Vala Tip: Don’t add an already implemented interface to a derived class

While implementing GSVG, I had to add DomDocumentType to a DomDocument, so your renderer will recognize it is a SVG document, so I use following:

class GomNode : DomNode

class GomDocumentType : GomNode, DomNode

Note GomNode is implementing DomNode, so yo don’t need to add that interface in its derived classes GomDocumentType.

Hours after a set of debugs, I found you should don’t do that, because it produce a infinite cycle, leading to a crash.

So, your GomDocumentType should be:

class GomDocumentType : GomNode

Now GomDocumentType is a DomNode too.

Why GSVG

At least for me, no SVG API is available as GObject API, so you have to deal with libxml2 library. On the other hand renderers are librsvg, but lacks API for SVG creation and Edition.

W3C has published an API specification version 1.1 for SVG and describes how renderers should interpret this XML document. W3C SVG depends on DOM as API to access XML documents.

While developing PLogic, I realized to have no way to create graphics for it, so take a look at GXml to dinamically generate SVG based on a logic diagram, using a GObject API to access XML documents. But no GObject API exists for SVG creation and edition.

Implement W3C API, is a matter to use OOP, like Java or C#, but a pain if you want to use pure C and GObject. So I choose Vala, because its syntaxis is very similar to W3C API and C#, making really easy to implement the API in a set of interfaces first and then in classes implementing those interfaces.

GXml lacks a good DOM support, due to liminations on libxml2 it relays on. So I started to implement a new pure GObject based set of classes, using libxml2 just for parsing and writting; they simple overcomes libxml2 limitations and now implements most DOM4,  whitout transformations.

GSVG now uses GXml’s GomNode classes series, improving time to implement features in W3C specifications.

GSVG have pushed GXml ahead, by exposing bugs and requesting new DOM4 implementations in order to provide useful SVG documents.

Support more than one Build System

I’ve tried to add Meson build system to MyHTML, but fail. They prefer the one is used today. That’s OK.

Support two build systems increase burden on project maintenance, this is the main reason to reject my pull request and is OK. As for GXml, we have both Autotools and Meson. I’m trying to keep both in sync, as soon as a new file is added, but you may forget one or the other.

While I use GXml on my Windows programs, I need to make sure it will work properly out of the box, like Autotools does, before to remove the later.

The main problem about Meson is: it is moving a lot, with new features added in resent versions; so they are not immediately available, i.e. in Windows through MSYS2. Just need to wait until Meson declares a LTS, or something like that, and is included by default in distributions and in MSYS2.

Extra effort to maintain both build systems makes the difference in my Vala development procedure: Code, Code Test, Run Test, Fix Code, Confirm Test Pass. This process have gained a boost in productivity, because Meson compile time and the way to run tests; so less time compiling, means more time Coding.

Last time I’ve tried to use Meson to create a release tar ball, fails. This is stopping me to switch or use it almost exclusively for development and release.

Vala 0.38 released

Vala’s maintainers have worked really hard this development cycle. They have fixed lot of new bug reports in a very short time.

One of the biggest change is merging Valadoc as part of Vala. So, now valadoc documentation utility will see official releases from the times to come, avoiding “drivers” for different Vala’s compiler versions.

Vala bindings for external projects, have been updated, including all resent Gtk+ 4.0 unstable API; bindings for javascriptcoregtk-4.0 have been added. All GIR based bindings, are generally updated each release, but this projects should provide Vala bindings generation by them selfs, so they can distribute along with any new release.

For details visit NEWS at Vala’s repository.

Vala 0.38 progress

Maintainers behind Vala compiler, have been busy this cycle.

Vala has a continuous integration machinery, so new features are added to staging branch before to land in master, in order to check some Vala projects can be compiled and their unit tests pass.

Expect more Vala improvements, fixes and backports to 0.34 and 0.36, so you can use them in your favorite distribution.

While 0.34 haven’t been officially declared as a long term support branch, it has received most fixes and backports, so stable releases, like Debian upcoming Strech, can provide reliable, stable and improved experience to developers targeting GNOME 3.22/Gtk+ 3.22 stable releases.

Vala’s bindings using GIR format are updated constantly and because them can be generated automatically from project’s sources, new API additions/changes land in master really fast. Is the case of GTK+ 4.0 unstable; allowing you to create testing/prototypes using latest hot GTK+’s new features, using Vala.

Highlights:

  • gobject-2.0: Add GLib.ParamSpecPointer
  • The CCode attribute ‘cname’ needs to be the canonical representation as it is expected in C. [Bug 731547]
  • gio-2.0: Use default ‘length = null’ for DataInputStream.read_line_utf8* [Bug 783351]
  • gvariant: Optimize (de)serialization of arrays with type-signature “ay” [Bug 772426]
  • Added –gresourcesdir switch, making easy to use gresg to autogenerate XML resource files in out of tree builds [Bug 783133]
  • Bindings updates
  • Gtk+ 4.0 bindings updates
  • codegen: Use *_free_full to free GLib.List, GLib.SList and GLib.Queue
  • Require and target GLib >= 2.40 [Bug 782005]
  • gdbus: Don’t leak nested HashTable on deserialization and make sure types derived from string are freed, e.g. ObjectPath [Bug 782719]
  • codegen: Fix finally blocks with async yields [Bug 741929]
  • compiler: Add –color=WHEN option following the format of other tools like diff and git-diff.
  • glib-2.0: Add quark() functions to errordomains
  • vala: Handle non-null in coalescing expression [Bug 611223]
  • glib-2.0: Add bindings for g_ptr_array_find/_find_with_equal_func()

gresg – an XML resources generator

For me, create GTK+ custom widgets is a very common task. Using templates for them, too.

Use GTK+ widgets defined by UI XML files, may be created by Glade, is a powerful feature.

Once you create your UI file, you should add it to a gresource XML file too, in order to use glib-compile-resources to compile an embed, if you wish, in your binaries.

Once your project is big enough, you may fall in a large gresource XML file. Regenerate compiled resources based on resources changes, can be tricky, and a hand work.

So I’ve created gresg, a tool to generate automatically an XML gresource file based in a list of files to be compiled with glib-compile-resources. This will help you to trigger a rebuild of compiled resources at any time you make changes in your files.

gresg, is written in Vala and uses GXml to generate XML resources files. As you can see in gresg’s repository, is a very small program.

If you are using Meson you can create a custom target to generate your XML resources, but you need this patch applied to Vala to take all its advantages and automatic re-build of resources.

Meson and GXml

After a call, Yannick has pushed a patch to add Meson build system to GXml. This is my first time using Meson and I really love it.

After a set of patches, I’ve managed to fix most installation and Unit Test integration.

Meson is well documented and provides a clean syntax.

Vala support is really good. In Autotools I’ve added some obscure rules to fix some old bugs. With Meson GXml has just a few ones, no obscure, commands in order to build Vala documentation and GObject Introspection binary files.

Meson exposes a bug in TDocument parsing, same test pass without error in Autotools. Using mesontest --gdb I was able to run tests in gdb, making things much convenient to debug than in Autotools, unless for the way I managed to debug in the past.

Meson is really fast! This will improve my development/tests/back to development processes, reducing time.

Next step is to find a way to get GXml compiled under Visual Studio, but first Gee needs to get Meson support too.

Vala 0.36 Released

This cycle Vala have received a lot of love from their users and maintainers. Users and maintainers, have pushed hard to get a lot of bug fixes in place, thanks to a lot of patches attached to bug reports.

List of new features an bug fixes are in NEWS file in repository. Bindings have received lot of fixes too, checkout them and see if you need a workaround.

Many thanks to all contributors and maintainers for make this release a big one.

Highlights

  • Update manual using DocBook from wiki.gnome.org as source [#779090]
  • Add support for array-parameters with rank > 1 in signals [#778632]
  • Use GTask instead of GSimpleAsyncResult with GLib 2.36/2.44 target [#763345]
  • Deny access to protected constructors [#760031]
  • Support [DBus (signature = …)] for properties [#744595]
  • Add [CCode (“finish_instance = …”)] attribute [#710103]
  • Support [HasEmitter] for vala sources [#681356]
  • Add support for the \v escape charactor [#664689]
  • Add explicit copy method for arrays [#650663]
  • Allow underscores in type parameter names [#644938]
  • Support [FormatArg] attribute for parameters
  • Ignore –thread commandline option and drop gthread-2.0 references
  • Check inferred generic-types of MemberAccess [#775466]
  • Check generic-types count of DelegateType [#772204]
  • Fix type checking when using generics in combination with subtype [#615830]
  • Fix type parameter check for overriding generic methods
  • Use g_signal_emit where possible [#641828]
  • Only emit notify of properties if value actually changed [#631267] [#779955]
  • Mark chained relational expressions as stable [#677022]
  • Perform more thorough compatibility check of inherited properties [#779038]
  • Handle nullable ValueTypes in signals delegates properly [#758816]

New GtkTester project

In my recent private developments, I need to create Gtk+ widgets libraries, and test them before use them in applications.

There are plenty of efforts to provide automated GUI testing, this is another one working in my case, I would like to share. It is written in Vala, is a GTK+ library with just one top window, you can attach your widget to test, can add test cases, check status and finish by calling asserts. Feel free to ask any thing you need or add issues, in order to improve it.

Sorry if the name is too GTKish and some one would like to change it to avoid any “Official Backup from GNOME”, which is not the case.

Hope to improve this library, adding more documentation in order to help others to use it, if they found useful.

Enjoy it at GitHub.