Daniel Espinosa

After years in development, mayor 6.0 version of GNOME Data Access library known as libgda has been released!

Vivien Malerba is the champion in the number of commits with 2071, when we see at history of libgda. The long stable 5.2 series, has been useful for many people out there, to create Database oriented applications. I’ve found libgda scripts written in Python very useful, when I need to import data from CSV format. Also programs written in other languages takes advantage of its features.

New 6.0 series, is a modernization in the heart of libgda, powered by Meson Build System, that push ahead its development, followed by a lot of fixes to use modern database providers like PostgreSQL, MySQL and SQLite. New features include a new API for Data Definition, allowing to hide some complexities to create database objects like tables and columns in GDA.

More providers could be better, but GDA now has less. Over time some interested developers may intent to fix current providers, but add new ones are not recommended.

GDA has an old implementation of the concept of GInterface for Providers, that will make hard to implement new ones. A new set of interfaces to implement new providers more easy is required.

GDA has now fixed lot of issues for multi-threading, that will help in some situations, thanks to its internals modernization to latest GObject definition like private structs.

GDA is written in C, making it portable and bindings to other languages is possible, one of them are the ones supported for GObject Introspection, like Vala, Python and JavaScript. Write scripts to access databases data using a Python script is possible.

What about the future, we need to port Providers to use GInterface so anyone can write providers easy. VDA has started prototyping this interfaces, making interesting to see how both projects can provider better tools for developers.

There are lot of things we will need to port to resent technologies, like GDA’s GTK widgets, they need to be ported to GTK 4.0 to provide a set of useful tools to access database’s data in widgets, leaving behind lot of bugs.

Hope GDA 6.0 be useful for developers as old 5.2 series is today, for many years in the future. I’m a believer of GNOME Technologies like GObject, because it is highly portable and so, reusable in many ways. GDA needs an overhaul in how it is developed, so any suggestion is welcome.

Many thanks to all contributors, translators and users, they have kept this software alive for a long time, requiring new features, issues reports and providing patches; they have found GDA useful, that make us an big incentive to fix and prepare GDA for the future.

Vala-Linter has merged my contribution to convert it to a library, while it has been integrated into GVls, the GNOME Vala Language Server, to diagnose coding style “errors” based on elementary’s one.

Just now found GVls is crashing, that wasn’t the case for a while, and that is because the way #ValaLinter uses @vala_lang’s library, I think. While the recovery is fast enough, I don’t like that situation.

Vala-Linter can be executed as a command line, that may is harder to use, but if the crashing is too frequent that may needs to be used by GVls to reduce the effect of crashing or just because a software crash is not good.

Vala’s library is used in its internal Unit Test System, that catch any crash or undesirable behavior. The harder part on having a Unit Type system is to find a way to reproduce an issue and fix it; then write the test to make sure it has gone away, while in the future the test will be executed on any change you made in your code, ensuring the issue is not back.

GVls was used by Vala’s library to fix lot of warnings; the warnings don’t prevent Vala’s compiler to work, but make GVls to fail on its Unit Test system, so an external code test was created, to test your entire code in GVls and find an issue not present in the tests cases in GVls, currently more than 90.

Vala’s library has an internal Code Visitor, GVls learn to not use, because there is a lot of use cases, hard to find the exactly one, producing crashing because you assume the code path has been tested, but that is not the case when you talk about compiling code; you are traversing the parsed code in a ways Vala’s library hasn’t been tested for, and that makes your code path hard to debug. GVls opted to not use Vala’s library Code Visitor and use its own, providing more control on how the code is visited and used to find your code symbols.

Yes, filing an issue will help, the problem is to find the exactly code that produce the crash. If you know a way to debug a subprocess that may will help: GNOME Builder uses GVls by default for Vala code, so running it in a debug mode, that can produce a backtrace when GVls’s server crash will help to identify and study it in order to fix it.

Vala-Linter integration in GVls is good from the point of view of functionality: you get code style warnings on your favorite IDE. But its use of Code Visitor, may will produce recurrent crashing. If so, I’ll consider to switch to call Vala-Linter’s command line and parse its output, to isolate GVls server.

Coding in Vala will be improved soon, when uncrustify integration is in place and GVls project will explore the way we can use it to provide on-type-formatting, to avoid server crash, if present, when Code Visitor is in use.

GXml is an Object Oriented implementation of DOM version 4, using GObject classes and written in Vala. Has a fast and robust serialization implementation from GObject to XML and back, with a high degree of control. After serialization, provides a set of collections where you can get access to child nodes, using lists or hash tables.

New 0.20 release is the first step toward 1.0. It provides cleaner API and removes old unmaintained implementations.

GXml is the base of other projects depending on DOM4, like GSVG an engine to read SVG documents based on its specificacion 1.0.

GXml uses a method to set properties and fill declared containers for child nodes, accessing GObject internals directly, making it fast. A libxml-2.0 engine is used to read sequentially each node, but is prepared to implement new ones in the future.

A new engine, still in development, tries to read nodes in a multi-threading environment, to speed up serialization.

GXml’s engine uses interfaces. That design has provided the opportunity to implement incrementally better new implementations, also holding more than one at the same time. The main ones, uses libxml-2.0 for sequential parsing to GObject, providing a fast process; the other, use libxml-2.0 parser to its own nodes, while provides bindings to a GObject classes, but limited to the features in libxml-2.0, limitations not present in the main implementation.

Through Vala, GObject can exploit all its potential, without the verbose C syntax but getting all its speed advantages.

Libraries written in Vala, will help powerful applications to be created and open new opportunities on innovations, while push up GNOME main technologies to be used outside Linux. Interest a side, is Vala’s features to create binaries not depending on GLib/GObject, open new opportunities to other markets.

Is not just help others to help you, is a matter of responsibility with Open Source Community. Your life have wonders and should change for better, so you will be lost opportunities or simple can’t work on your favorite open source project. Prepare your self to be a maintainer professor, change your mind for the beginning and help others, that is also a great contribution to open source software.

Be kind. Your potential contributors will take over when required. Making sure they have the abilities and use best practices in the project, is not just good for your project, is good for all others out there; they will use them to help other projects.

I suggest help others in each step they take to help, like:

1. Take care about your words. Words are powerful and will invite or reject potential helpers.
2. Help others to find resources on how write good issue reports, like the one in BSD.
3. Say thanks! your contributors will, even the ones not providing any useful information or code, take the time to use or read your project’s code, so they expend time, a valuable time.
4. Take the time to train your potential contributors, they will take compromise quickly if you are the maintainer and you help them; yes the time will never enough, but that is an inversion for the future.

If you want to be a maintainer, start by helping open source projects. There are a set of online resources you can use to make you better as a contributor and maintain your own software or take over the maintainership of your favorite software:

1. Fundamentals of Professional Open Source Management
2. Open Source Maintainer

Also, you can read Georges Stavracas’ experience on On Being a Free Software Maintainer, as an inspiration.

 

There is a case when you wish to use a custom widget, when you use the same set of widgets to handle data. When you create a Gtk.ListBoxRow you have to attach to it a Gtk.Widget, in that case, should be easiest if you just do

var w = new MyWidget ();
w.data = data;


Second line, sets your data to the widget’s data property, so the class initialize the UI with the provided data.

In other case, you have a Widget you can re-use in different parts of the UI, maybe showed up in a popup window. This is the process you can use to create such a widget.

Create a Widget in Glade

Yes, use Glade to create a template UI file for your new widget, for the example, lets start with this simple user interface:

Key aspects is set Composite in order to define it as a Widget class definition, make sure you set Class Name to the same C name of your class, that is include the namespace, in our example the namespace is My so the widget class name is set to MyWidget.

Define your Class

Now, make sure you setup the Widget class to  use the UI  you will use. Name all entries and buttons, or any other UI elements you plan to use in your custom widgets at UI design time; use that name in your source code as follows:

Gtk.Entry evalue1 will be linked to your entry with the Id evalue1 so you can use it as any other variable. To link, use [GtkChild] annotation and [GtkTemplate]  one for the class, setting the attribute uiwith the name of the UI file in the resources file; in this case defined as:

UI file linked to the widget class, should be embedded in your program as a GLib resource. Set prefix to the path for the resource resolution, useful to classify different resources.

If you misspell the UI file or the ChildWidge‘s name you’ll get the following kind of errors:

Make it Interactive

Lets take a view at the bmultiply variable, linked to a Gtk.Button with the id set to, and how above code connect to its clicked signal to perform an operation and set the result to the text of the Gtk.Entry with the Id eresult  just accessing the variables defined as private in the My.Widget.

Base for your Class

Above widget, is a subclass of  Gtk.Box so you should make sure your top level widget in the UI definition is of the same type, as you can see at the Glade screenshot. You can take any other base class, depending on the functionality you want to start from.

GNOME Vala Language Server, GVls, 0.14.0 has been released!

GVls has been integrated with GNOME Builder and now is its default Vala Language Server!

New features include:

* Completion: implementing filtering on partial words
* SourceView: on completion now provides just the position on code
* onChange: improved UTF-8 handling
* Server: can configure its defaults
* Server: request configuration after ‘initialized’ notification from the client
* Server: can handle configuration changes notifications
* Server: scan for all vala sources in workspace at initialization
* Server: can request configuration from Client
* Server: implemented diagnostics publication
* Server: can response both DocumentSymbol[] or SymbolInformation[] for textDocument/documentSymbol
* Server: search all symbols in a namespace
* Client: can provide build environment server configuration
* Client: expose Vala build settings at server initialization
* Client: can send configuration changes notifications
* Client: notify ‘initialized’ to the server
* Client: notify ‘didChangeConfiguration’ to the server
* ValaConfiguration: new Vala build environment settings object
* General fixes on Variant deserialization
* GNOME Builder: fixes to handle its requests
* API changes due to fixes

Current features will help to integrate GVls with other LSP clients like VSCode, implementing a Client to connect to GVls’s Server.

Now opening for 0.16 release cycle will include a refactoring of the server to share more common VAPI files like glib-2.0 and others.

Memory Footprint

For small projects like GVls or GXml, you have descent 150MB memory footprint and good response time, but for large projects like Vala, that will be increased to 6.9GB!!! unacceptable in my small resources machine, so we need to take action here!

Database driven storage can help on! Store parsed symbols in a database will help to share common used VAPI and reduce footprint on large projects with a possible impact on Server’s response time. My first is to use VDA to get access to simple SQLite or more powerful PostgreSQL servers (may you want to implement any other native or use existing GDA backend), some of the work has been made by Anjuta, so we can share databases or data model or make GVls the server for Anjuta.

There is a Language Server Specification (LSP) published by Microsoft. Is a set of interfaces to manage information about a set of source files in an specific programming language, like symbols, classes, properties, variables, enumerations, and so on.

LSP uses JSON to communicate between a Server and a Client, so all interfaces must be translated to its counterpart in JSON before to transmit over. The communication is bidirectional, so when a request is sent from one side to another, a response is sent back with the result.

An LSP Client can used by Source Code Editors, like GNOME Builder or elementary’s code, or Anjuta, in order to provide specific services to the coder, like completion, goto symbol definitions, diagnostics, code formatting and others. A Client request all above services from a Server. This way, a source code editor can support multiple programming languages, just implementing an LSP Client and connect to an LSP Server.

Client and Server should cooperate in order to provide power full tools to the user. Some times one or other, provides limitations or the cooperation requires to changes in the source code editor design.

Anjuta for example, has implemented a database to track symbols definitions, this is no necessary as an LSP Client, because the Server should track them for you. Client just need to request if a symbol is found and get back its type and children (a very common concept on Object Oriented Programming Languages like Vala).

A Server requires lot of information from the Client, like the root directory and compilation flags used, so it can provide reliable diagnostics and, like in Vala, the --pkg switches in use so it can find the namespaces in use for both, completion and diagnostics.

Vala has its own Language Compiler and as many others, creates a tree of structured information about the code in order to translate it to C and then use another compiler to transform the resulted code to machine code. The Vala’s tree lot of the information we need to implement an LSP.

Recently GVls was accepted by GNOME Builder, as The Vala Language Server for source editing. It provides completion, goto definition and diagnostics, with more services coming.

There are improvements and new implementations in GVls to work on and hope a Client for Microsoft Visual Studio Code will be implemented soon.

Vala will be impacted by GVls for sure, in order to improve diagnostics messages, symbol resolution, completion and so on, just need to push it to its limits and find new opportunities for improvements.

GNOME Builder, has gained some new LSP implementations while integrating GVls and hope both projects found a way to provide more powerful features for Object Oriented Programmers. One area to work on, is on Vala Source Code Highlighting, in order to help programmers to know if an used symbol is correctly spelled by seeking for in the Server’s symbols database. Diagnostics is other one, where we need a way to introspect the Build System, in order to find Vala’s compiler’s switches in use, so we can improve error or warning messages.

JSON to/from GObject

In order to implement LSP in GVls, a new GVls.VariantObject interface to be implemented by any GObject class. It provides any GObject the capability to be converted to/from its GLib.Variant representation. In GVls we use JSON-GLIB, so a GLib.Variant is converted to JSON and back, so now you can use any JSON based service to serialize and parse information using, for example, JSON-RPC which in turn is the one used to implement a LSP Client and Server communication.

Recently GNOME Calculator has gained a library for math expression parsing and calculation, as a parallel effort to the one used internally for the application, called GCalc.

While GCalc allows to take a string to create an object oriented representation of it and can perform multi-precision calculations, this feature was unable to be used outside, just VDA is using it for math expression parsing, but no-more, yet.

In order to expose GCalc features to user oriented applications, now a new library called GCi was added. GCi provides, for now, a controller for GTK Entries.

GTK Entries controller in GCi, can add calculation features to your Entry. Once you have created a GCi.EntryController and set its entry property to the one in your UI application, a secondary icon using a calculator is added, while allows your users to  write =8*2+1, hit enter or click on the secondary icon and the math expression is replaced by its calculation result (and yes you should use = in order to establish it as a math expression to replace).

Vala Data Access library has reached a 0.90 Beta 1 release.

VDA provides a set of interfaces to wraps database connection, execution of SQL commands and access to returned values of the queries. Read the previous introduction post.

This version supports:

• First Beta version of VDA
• Native implementation for PostgreSQL
• GDA implementation with support for SQLite and PosgreSQL
• Basic SqlValue implementation and conversion between values
• Basic SQL commands parsing support for SELECT, INSERT, DELETE, UPDATE, using parameters in order to provide the values required to execute it
• Object Oriented API for SQL commands: SELECT, INSERT, DELETE, UPDATE
• Support for Parameters in queries using GDA declaration syntax
• SQL command parsing use GCalc from GNOME Calculator
• Async queries execution
• Support to map Row’s values to GObject based classes’ properties using DataObject interface
• Support for row modification/insert using DataObject interface
• DataObject supports Vda.SqlValue properties, both generic or specific
• DataObject support conversion between Vda.SqlValue values and between basic properties’ types like string, int, double, float, bool

No pre-releases was made, because the API was changed while implementing database providers and included most of the interfaces to implement for better more powerful features.

VDA return Vda.SqlValue objects when you access a Table’s row’s column’s value; the provider is responsible to create it when is requested, so no overhead is present for SQL queries execution.

I think this is new, so I’ll describe a new technique used in GXml to parse a large set of nodes in an XML document.

Parsing Large XML documents

If you have a large XML document, with a root with a number of child nodes, the standard technique is read all of them, including the child’s children ones, to create the XML tree. This process can take a while.

New on-the-fly parser

GXml now has a new custom parser called StreamReader used to read the root element and its children, but without any attribute and without any child’s children; the attributes and the children’s children are stored in a string on-the-fly in order to read the document almost at the same time it is read from the IO stream, for the root and for each children, improving the loading time of large XML documents up to 400% times faster than the previous technique already present in GXml.

On-the-fly-post-parsing

By using this On-the-fly-post-parsing technique, You can’t access the child’s children or the root’s attributes immediately after first read, you have to parse it from a temporally location in the GXml.Element class, using the new GXml.Element.parse_buffer() method, this one use the standard method, already present in GXml, to parse the root’s properties and the children’s children. When GXml.Element.parse_buffer() is called over the root, all children’s children are parsed recursively, but you can choose to parse just one of the root’s child, making a really convenient technique when you need just one root’s child node in a large XML document.

Multi-threading parsing

Currently GXml.Element.parse_buffer_async(), when called on root’s element, uses GLib.ThreadPool to parse each child in a different thread each and uses as many as threads are usable (less one) in your system. The expected behavior is getting a parse boost over the standard technique using in GXml: Xml.TextReader from the veteran libxml2 library running over just one thread. Currently a standard time parsing is provided when GXml.Element.parse_buffer_async() is called on document’s root, this maybe is a limitation on libxml2, because we have lot of Xml.TextReader running at the same time parsing element’s children; or a limitation on GLib.ThreadPool. Maybe the solution is a step away.