Re: Consider the maintainer

I’ve read this LWN article: Consider the maintainer. It was a great read, and I want to share my thoughts, from my experience on being a maintainer (or helping the maintenance) of several GNOME modules.

GNOME has a lot of existing code, but let’s face it, it has also a lot of bugs (just look at bugzilla, but the code also contains a lot of not-yet-reported bugs). For a piece of software to be successful, I’m convinced that it has to be stable, mostly bug-free. Stability is not the only property of a successful software, but without it it has way less chance to be successful in the long run (after the hype wave is gone and the harsh reality resurfaces).

There is a big difference between (a) writing a feature, but in reality it’s full of bugs, and (b) writing the same feature, but “mostly bug-free” (targeting bug-free code). It certainly takes the double of time, probably more. The last 10% of perfection are the most difficult.

Paolo Borelli likes to explain that there are two kinds of developers: the maintainers and the developers who prefer to write crazy-new-experimental features (with a gray scale in-between). It is similar with the difference between useful tasks vs interesting tasks that I talked about in a previous blog post: some useful tasks like writing unit tests are not terribly interesting to do, but I think that in general a maintainer-kind-of-developer writes more tests. And Paolo said that I’m clearly on the maintainer side, caring a lot about code quality, stability, documentation, tests, bug triaging, etc.

Reducing complexity

The key, with a lot of existing but not perfect code, is to reduce complexity:

  • Improving the coding style for better readability;
  • Doing lots of small (or less-small) refactorings;
  • Writing utility classes;
  • Extracting from a big class a set of smaller classes so that the initial class delegates some of its work;
  • Writing re-usable code, by writing a library, and documenting the classes with GTK-Doc;
  • Etc.

Even for an application, it is useful to write most of the code as an internal library, documented with GTK-Doc. Browsing the classes in Devhelp is such a nice way to discover and understand the codebase for new contributors (even if the contributor has already a lot of experience with GLib/GTK+).

Another “maintainer task” that I’ve often done: when I start contributing to a certain class, I read the whole code of that class, trying to understand every line of code, doing lots of small refactorings along the way, simplifying the code, using new GLib or GTK+ APIs, etc. When doing this exercise, I have often discovered (and fixed) bugs that were not reported in the bug tracker. Then to achieve what I wanted to do initially, with the much better knowledge of the code, I know how to do it properly, not with a quick hack to do the minimal amount of change that I sometimes see passing. As a result the code has less bugs, there is less chance to introduce new bugs, and the code is easier to understand and thus more maintainable. There is no secrets, it takes more time to do that, but the result is better.

Some books that were very useful to me:

Of course I didn’t read all those books at once, practicing is also important. I nowadays read approximately one computing science book per year.

About new contributors and code reviews

When I started to contribute to GtkSourceView several years ago, I had already developed a complete LaTeX editor based on GtkSourceView (by myself), read several of the above books (most importantly Code Complete) and applied what I learned. I had already a lot of experience with GTK+. So starting to contribute to GtkSourceView was easy, my patches were accepted easily and I think it was not too much work for the reviewers. I then became a co-maintainer.

Contrast this with all those newbies wanting to contribute to GNOME for the first time, without any experience with GLib/GTK+. They don’t even know how to contribute, how to compile the code, they probably don’t know well the command line or git, etc. So if a maintainer wants to help those newcomers, it takes a lot of time. I think this is partly a problem of documentation (that I’m trying to solve with this guide on GLib/GTK+). But even with good documentation, if the new contributor needs to learn for the first time GTK+, it will require too much time for the maintainer. What I would suggest is for newcomers to start by writing a new application on their own; for that a list of ideas of missing applications would be helpful.

This is maybe a little controversial, but the talk Consider the maintainer was also controversial, by suggesting for instance: “Maintainers should be able to say that a project is simply not accepting contributions, or to limit contributors to a small, known group of developers.”

When a company wants to hire a developer, they can choose the best candidate, or if no candidates fit they can also choose to keep the existing team as-is. In Free Software, anyone can send a patch; sometimes it takes a lot of time to explain everything and then after a short time the contributor never comes back. Remember also the well-known fact that adding people to a late project makes it later (usually, but there are exceptions).

Another interesting glimpse, from Hackers and Painters (Paul Graham):

I think this is the right model for collaboration in software too. Don’t push it too far. When a piece of code is being hacked by three or four different people, no one of whom really owns it, it will end up being like a common-room. It will tend to feel bleak and abandoned, and accumulate cruft. The right way to collaborate, I think, is to divide projects into sharply defined modules, each with a definite owner, and with interfaces between them that are as carefully designed and, if possible, as articulated as programming languages.

Other topics

I could talk about other topics, such as the lack of statistics (I don’t even know the number of people executing the code I write!) or trying to avoid sources of endless maintenance burden (example: GtkSourceView syntax highlighting definition files, the maintenance could clearly be better distributed, with one maintainer per *.lang file). But this blog post is already quite long, so I’ll not expand on those topics.

In short, there is clearly matter for thoughts and improvements in how we work, to get more things done.

Posted in GtkSourceView, LaTeXila, Library development, Me Myself and I, Programming, Thoughts | 5 Comments

Spell-checking for GtkEntry in gspell

It’s done! Everything that I wanted to do initially for the fundraising of gspell is implemented (for the milestone 1).

The main steps were:

  1. Basic infrastructure, insert underlines to misspelled words.
  2. Add the context menu, and have common code between GtkTextView and GtkEntry.
  3. Do not check the word currently typed. Ditto, have common code between GtkTextView and GtkEntry, to define the policy at only one place.
  4. Better word boundaries: take into account apostrophes and dashes.
  5. Take care of a few other details, like forcing to disable spell-checking when the GtkEntry is in password mode.

The fundraising has a second milestone with “Various Other Improvements”, which is currently at 17%, so I’ll do a few more small tasks. For example fixing bug #772406 (a unit test fails with aspell, while it works fine with hunspell, the kind of problems that arise when multiple backends can be used and work slightly differently) and what is described in bug #761921 comment #1 (a first possible step for a better Windows support).

Posted in gspell | 6 Comments

Smooth transition to new major versions of a set of libraries

With GTK+ 4 in development, it is a good time to reflect about some best-practices to handle API breaks in a library, and providing a smooth transition for the developers who will want to port their code.

But this is not just about one library breaking its API. It’s about a set of related libraries all breaking their API at the same time. Like what will happen in the near future with (at least a subset of) the GNOME libraries in addition to GTK+.

Smooth transition, you say?

What am I implying by “smooth transition”, exactly? If you know the principles behind code refactoring, the goal should be obvious: doing small changes in the code, one step at a time, and – more importantly – being able to compile and test the code after each step. Not in one huge commit or a branch with a lot of un-testable commits.

So, how to achieve that?

Reducing API breaks to the minimum

When developing a non-trivial feature in a library, designing a good API is a hard problem. So often, once an API is released and marked as stable, we see some possible improvements several years later. So what is usually done is to add a new API (e.g. a new function), and deprecating an old one. For a new major version of the library, all the deprecated APIs are removed, to simplify the code. So far so good.

Note that a deprecated API still needs to work as advertised. In a lot of cases, we can just leave the code as-is. But in some other cases, the deprecated API needs to be re-implemented in terms of the new API, usually for a stateful API where the state is stored only wrt the new API.

And this is one case where library developers may be tempted to introduce the new API only in a new major version of the library, removing at the same time the old API to avoid the need to adapt the old API implementation. But please, if possible, don’t do that! Because an application would be forced to migrate to the new API at the same time as dealing with other API breaks, which we want to avoid.

So, ideally, a new major version of a library should only remove the deprecated API, not doing other API breaks. Or, at least, reducing to the minimum the list of the other, “real” API breaks.

Let’s look at another example: what if you want to change the signature of a function? For example adding or removing a parameter. This is an API break, right? So you might be tempted to defer that API break for the next major version. But there is another solution! Just add a new function, with a different name, and deprecate the first one. Coming up with a good name for the new function can be hard, but it should just be seen as the function “version 2”. So why not just add a “2” at the end of the function name? Like some Linux system calls: umount() -> umount2() or renameat() -> renameat2(), etc. I admit such names are a little ugly, but a developer can port a piece of code to the new function with one (or several) small, testable commit(s). The new major version of the library can rename the v2 function to the original name, since the function with the original name was deprecated and thus removed. It’s a small API break, but trivial to handle, it’s just renaming a function (a git grep or the compiler is your friend).

GTK+ timing and relation to other GNOME libraries

GTK+ 3.22 as the latest GTK+ 3 version came up a little as a surprise. It was announced quite late during the GTK+/GNOME 3.20 -> 3.22 development cycle. I don’t criticize the GTK+ project for doing that, the maintainers have good reasons behind that decision (experimenting with GSK, among other things). But – if we don’t pay attention – this could have a subtle negative fallout on higher-level GNOME libraries.

Those higher-level libraries will need to be ported to GTK+ 4, which will require a fair amount of code changes, and might force to break in turn their API. So what will happen is that a new major version will also be released for those libraries, removing their own share of deprecated API, and doing other API breaks. Nothing abnormal so far.

If you are a maintainer of one of those higher-level libraries, you might have a list of things you want to improve in the API, some corners that you find a little ugly but you never took the time to add a better API. So you think, “now is a good time” since you’ll release a new major version. This is where it can become problematic.

Let’s say you released libfoo 3.22 in September. If you follow the new GTK+ numbering scheme, you’ll release libfoo 3.90 in March (if everything goes well). But remember, porting an application to libfoo 3.90/4.0 should be as smooth as possible. So instead of introducing the new API directly in libfoo 3.90 (and removing the old, ugly API at the same time), you should release one more version based on GTK+ 3: libfoo 3.24. To reduce the API delta between libfoo-3 and libfoo-4.

So the unusual thing about this development cycle is that, for some libraries, there will be two new versions in March (excluding the micro/patch versions). Or, alternatively, one new version released in the middle of the development cycle. That’s what will be done for GtkSourceView, at least (the first option), and I encourage other library developers to do the same if they are in the same situation (wanting to get rid of APIs which were not yet marked as deprecated in GNOME 3.22).

Porting, one library at a time

If each library maintainer has reduced to the minimum the real API breaks, this eases greatly the work to port an application (or higher-level library).

But in the case where (1) multiple libraries all break their API at the same time, and (2) they are all based on the same main library (in our case GTK+), and (3) the new major version of those other libraries all depend on the new major version of the main library (in our case, libfoo 3.90/4.0 can be used only with GTK+ 3.90/4.0, not with GTK+ 3.22). Then… it’s again the mess to port an application – except with the following good practice that I will just describe!

The problem is easy but must be done in a well-defined order. So imagine that libfoo 3.24 is ready to be released (you can either release it directly, or create a branch and wait March to do the release, to follow the GNOME release schedule). What are the next steps?

  • Do not port libfoo to GTK+ 3.89/3.90 directly, stay at GTK+ 3.22.
  • Bump the major version of libfoo, making it parallel-installable with previous major versions.
  • Remove the deprecated API and then release libfoo 3.89.1 (development version). With a git tag and a tarball.
  • Do the (hopefully few) other API breaks and then release libfoo 3.89.2. If there are many API breaks, more than one release can be done for this step.
  • Port to GTK+ 3.89/3.90 for the subsequent releases (which may force other API breaks in libfoo).

The same for libbar.

Then, to port an application:

  • Make sure that the application doesn’t use any deprecated API (look at compilation warnings).
  • Test against libfoo 3.89.1.
  • Port to libfoo 3.89.2.
  • Test against libbar 3.89.1.
  • Port to libbar 3.89.2.
  • […]
  • Port to GTK+ 3.89/3.90/…/4.0.

This results in smaller and testable commits. You can compile the code, run the unit tests, run other small interactive/GUI tests, and run the final executable. All of that, in finer-grained steps. It is not hard to do, provided that each library maintainer has followed the above steps in the good order, with the git tags and tarballs so that application developers can compile the intermediate versions. Alongside a comprehensive (and comprehensible) porting guide, of course.

For a practical example, see how it is done in GtkSourceView: Transition to GtkSourceView 4. (It is slightly more complicated, because we will change the namespace of the code from GtkSource to Gsv, to stop stomping on the Gtk namespace).

And you, what is your list of library development best-practices when it comes to API breaks?

PS: This blog post doesn’t really touch on the subject of how to design a good API in the first place, to avoid the need to break it. It will maybe be the subject of a future blog post. In the meantime, this LWN article (Designing better kernel ABIs) is interesting. But for a user-space library, there is more freedom: making a new major version parallel-installable (even every six months, if needed, like it is done in the Gtef library that can serve as an incubator for GtkSourceView). Writing small copylibs/git submodules before integrating the feature to a shared library. And a few other ways. With a list of book references that help designing an Object-Oriented code and API.

Posted in GtkSourceView, Library development, Programming, Thoughts | 7 Comments

gspell and LaTeXila – progress report

In September I’ve launched two small fundraisings on gspell and LaTeXila. The two goals are now reached, thanks!

I’ve started working on those two projects, here is a progress report.

gspell – adding GtkEntry support

The basic infrastructure is there, and red wavy underlines are inserted to highlight misspelled words. The code is written in a way that it is unit-testable, and unit tests have been written.

Statistics so far:

21 files changed, 1921 insertions(+), 14 deletions(-)

Next steps:

  • The context menu (right-click menu).
  • Do not check the word currently typed.
  • Better word boundaries, to take into account apostrophes and dashes.

For all those next steps, the idea is to have common code between the GtkTextView and GtkEntry support.

LaTeXila – port to GAction/GMenu?

This progress report is a bit more technical.

To create the menu and toolbars, LaTeXila still uses the deprecated GtkUIManager and GtkAction. What I’ve done this week is investigation work, to figure out if it is possible to use GMenu (the new way menus are now usually created). GAction is great and will be used, but for GMenu it is more complicated.

The conclusion is that it will not be possible to use GMenu in LaTeXila. Because in LaTeXila, when hovering a menu item, a longer description is displayed in the status-bar – which makes the user interface self-discoverable – and this is not possible with GMenu (see this comment in bugzilla for more details). Thankfully GTK+ has a more basic way to create menus, with GtkMenuBar, GtkMenuItem etc. So that’s what LaTeXila will use.

Another reason to create a GtkMenuBar manually is to be able to add a sub-menu with the list of recently used files, with GtkRecentChooserMenu. This doesn’t exist as a GMenu, it would need to be implemented.

Also, the nice thing about GtkUIManager is that the information was encoded just once for both the menus and the toolbars. For a certain menu or toolbar item, the needed information is: the short description, the long description, the icon, the keyboard shortcut and the function to call when the action is activated. With GMenu, some of the information needs to be duplicated for creating a toolbar. So that’s another reason why not using GMenu (at least not directly).

So what I will do is to create a simplified GtkUIManager based on GAction. In other words, store the information just once, and have a convenient way to create menus and toolbars. To create a menu or toolbar item, all of what will be needed is to provide the action name; it will fetch the required missing information from a central store.

Posted in gspell, LaTeXila | 6 Comments

gspell and LaTeXila fundraisings – thanks!

The gspell fundraising has reached its initial goal! So thanks a lot for your support!

Expect GtkEntry support in the next version of gspell, which is planned for March 2017.

I’ve added a second milestone for the gspell fundraising, because there are a lot of other possible things to do.

The LaTeXila fundraising is going well too, it is currently at 80%. So thanks to the people who donated so far!

If you write documents in LaTeX, and care about having a good LaTeX editor, well maintained and well integrated to GNOME, then consider donating to LaTeXila ;) There will hopefully be other milestones in the future, for example to improve the auto-completion (e.g. complete the label in \ref commands), or implementing a full-screen mode.

Posted in gspell, LaTeXila | Comments Off on gspell and LaTeXila fundraisings – thanks!

Two Small Fundraisings – on gspell and LaTeXila

We live in a world where it’s increasingly possible to have a part-time job and being paid for other small tasks on the side; think about Uber or airbnb.

I have a half-time job, and I care about Free Software. So the natural thing to do for me is to find ways to be funded for the contributions I do.

That’s why I was really pleased to hear Alexander Larsson, at the end of his talk at GUADEC about Flatpak:


[…]
And also an interesting thing – I think actually super important – is payment. I want people to be able to pay for Free Software.

Which was met with applause. But unfortunately such support in Flatpak and GNOME Software are not going to happen anytime soon, maybe in a few years with some hope.

In the meantime, I’m launching two small fundraisings!

If this is successful, the idea is to add further milestones, after the work is done on the first one.

gspell

gspell is a library that I created last year to share the spell-checking code between gedit and LaTeXila. The code comes from the gedit spell plugin, as such there is only the support for GtkTextView. The goal of the fundraiser is to add support for GtkEntry, a single line text entry field.

Go to the gspell fundraising page if you’re interested!

LaTeXila

GTK+ 3.22, which will be released this Wednesday, will be the last GTK+ 3 stable version (see this announcement). After that, the deprecated functionality of GTK+ 3 will be removed. LaTeXila is not a new application, it is developed since 2009. A fundraising on a new piece of software can concentrate on features. For LaTeXila, what is more important is that it is well maintained, to be still relevant in the 2020’s. So the fundraiser is to make the code ready for GTK+ 4, at least as a next step.

Code maintenance is not really what we can call interesting, but it is useful. We can see this interesting/useful dichotomy quite often in the computer science world: for a researcher, is proof of correctness interesting? Yes. But in practice it is rarely applied. Is writing unit tests interesting? No, but it is very useful.

Go to the LaTeXila fundraising page if you’re convinced ;)

Posted in gspell, LaTeXila | 8 Comments

Announcing Gtef, an incubator for GtkSourceView

Gtef – the acronym for “GNOME Text Editor Framework” – is a new library that eases the development of GtkSourceView-based text editors and IDEs. It can serve as an incubator for some GtkSourceView features.

See the Gtef repo on GitHub for more details. In particular, Gtef has an interesting way to handle API (un)stability. The Gtef Roadmap is alongside the GtkSourceView one.

Posted in Gtef, GtkSourceView | 2 Comments

Using the GtkSourceView API to write scripts that manipulate text

In the gnome-c-utils repository, I wrote some scripts that use the GtkSourceView library.

When a script needs to read some text, search something in it, and possibly edit the content, then having a GtkTextBuffer is really convenient.

GtkTextBuffer is good at navigating through text, with all the GtkTextIter functions. The content can be edited easily. And the GtkSourceView library adds interesting features like regex search & replace and file loading and saving with character encoding handling (GtkTextBuffer accepts only valid UTF-8).

So, that’s it. I just wanted to share that I find it convenient to write scripts with GtkSourceView (but maybe because I speak the GtkTextView/GtkSourceView API fluently).

Posted in GtkSourceView | Comments Off on Using the GtkSourceView API to write scripts that manipulate text

Semi-Object-Oriented Programming in C

Although I said last year that I’m not the right person to write a book about developing GLib/GTK+ applications and libraries, I continue slowly but surely.

There is now a new section called “Semi-Object-Oriented Programming in C”. This serves as a nice warming up for GObject.

Download the PDF:

The GLib/GTK+ Development Platform (version 0.5)

See also the NEWS file to know what changed since my last blog post on the subject.

Any comments or patches are welcome (the Git repo is here).

Posted in book | 2 Comments

Thoughts on the Linux Mint X-Apps forks

You may be aware that Linux Mint has forked several GNOME applications, either directly from GNOME (Totem -> Xplayer, Evince -> Xreader, Eye of GNOME -> Xviewer), or indirectly via MATE (gedit -> pluma -> XEd).

GNOME is like the Debian of the Linux desktops. But is it a good thing? In the current state of the code, I don’t think so and I’ll explain why, with a solution: creating more shared libraries.

At the end of the day, it’s just a matter of design and usability concerns. We can safely say that the main reason behind the forks is that the Linux Mint developers don’t like the new design of GNOME applications with a GtkHeaderBar.

And there are perfectly valid reasons to not like headerbars. For gedit for example, see the list of usability regressions at this wiki page.

Currently the trend is GtkHeaderBar, but what will it be in 5 years, 10 years? Let’s face it, GNOME is here just following the new trend that came with smartphones and tablets.

So, a GNOME application developer needs to know that:

  • A GUI is an ever-changing thing, exactly like the clothes that you bought last year are already obsolete, right?
  • When the GUI changes too much, other developers don’t like it and fork the project. For valid reasons or not, this doesn’t matter.

The four X-Apps forks account for roughly 200k lines of code. In the short-term it works, Linux Mint has apps with a traditional UI. But hey, porting the code to GTK+ 4 will be another beast, because the four X-Apps still use the deprecated GtkUIManager and GtkAction APIs, among other things.

But when we look at the codebase, there are a lot of code that could be shared between a GNOME app and its fork(s). So there is a solution: creating more shared libraries. The shared libraries would contain the backend code, of course, but also some basic blocks for the UI. The application would just need to glue things up together, assembling objects, binding GObject properties to GSettings, create the main GtkWindow and a few other things.

The difference would be that instead of forking 200k lines of code, it would be forking maybe 20k lines, which is more manageable to maintain in the long term.

In the case of gedit, making its code more re-usable is exactly what I do since several years, but for another reason: being able to create easily specialized text editors or small IDEs.

Beside avoiding code duplication, creating a shared library has the nice side effect that it is much better documented (usually), and with an API browser like Devhelp, it’s a breeze to discover and understand a new codebase, it permits to have a nice overview of the classes. It’s of course possible to have such documentation for application code, but in practice few developers do that, although it would be a big step towards lowering the barrier to entry for newcomers.

When untangling some code from an application and putting it in a shared library, it is also easier to make the code unit testable (and unit tested!), or at least write a mini interactive test in case of frontend code. Making the code more stable, getting closer to bug-free code and thus more successful software.

Developing a shared library doesn’t necessarily mean to provide backward compatibility during 10 years. Nothing prevents you from bumping the major version of the library every 6 months if needed, making the new version parallel-installable with the previous major versions. So that applications are not forced to update the code when there is an API break.

But creating a library is more difficult, API design is hard. But in my opinion it is worth it. GNOME is not only a desktop environment with an application suite, it is also a development platform.

Posted in gedit, gspell, GtkSourceView, Library development, Programming, Thoughts | 12 Comments