Happenings in GNOME

I made another weird side project while unemployed. In fact I’ve wanted it for a while but once I learned that “Rebonjour” is the word for “hello again” I just had to finish the library.

librebonjour is an asynchronous DNS-SD and mDNS client library for GLib applications. Or, more practically, it is a small GObject API over the two local service-discovery providers you are likely to find on a Linux system: Avahi and systemd-resolved.

It does not link against either of them. It only talks to them over D-Bus.

The reason for that is mostly boring, which is usually where the useful things are. Applications should not need to care if a machine has Avahi running, or if it is using systemd-resolved for mDNS. They should be able to discover a service, resolve it, maybe advertise something, and get on with whatever they were actually trying to do.

So RebonjourClient selects a backend internally. If org.freedesktop.Avahi is available on the system bus, it uses Avahi. If not, it falls back to systemd-resolved’s org.freedesktop.resolve1 API. If neither is around, availability checks fail like you would expect.

The public API stays the same either way.

What It Does

There are three common things I wanted to make pleasant.

First, one-shot discovery. Ask for the service types in local, ask for instances of something like _ipp._tcp, then resolve one of those instances into addresses and TXT metadata.

Second, browser-style discovery. A RebonjourBrowser owns a stable GListModel of RebonjourService objects. That fits nicely into GTK code because the model object can stay the same while the contents change underneath it.

Third, registration. You can describe a local service with RebonjourServiceDescription, register it, and keep the returned RebonjourRegistration alive for as long as the service should be advertised.

Resolving a service gives you a RebonjourResolvedService. That contains the SRV result, TXT data, priority, weight, and a model of RebonjourEndpoint objects. The endpoints hold the GSocketAddress you would actually use to connect.

Why Two Backends

Avahi is the nicer backend for browsing. Its D-Bus API gives you long-lived browser objects and emits signals when services appear and disappear. That maps very naturally to GListModel changes.

systemd-resolved is different. It has useful DNS-SD and mDNS operations over D-Bus, but the browsing side is lookup-based. That means you can ask what is there, but you do not get the same live add/remove signal stream that Avahi provides.

I did not want applications to have to care about that distinction unless they really want to. So the browser has auto-refresh and refresh-interval properties. With Avahi, auto-refresh is effectively harmless because the model is already live. With systemd-resolved, it starts an internal refresh loop and updates the model for you.

It is not magic. It is just putting the backend-shaped unpleasantness in one place so application code can stay boring.

Asynchronous with libdex

The whole thing is built on libdex. Anything that might touch D-Bus or the network returns a DexFuture.

That means construction, availability checks, service-type lookup, instance lookup, resolving, registration, browser refresh, and unregistering are all future-based. If you are already writing fiber-style code with libdex, the API fits into that directly:

g_autoptr(RebonjourClient) client = NULL;
g_autoptr(GListModel) services = NULL;
g_autoptr(GError) error = NULL;

if (!(client = dex_await_object (rebonjour_client_new (), &error)))
  g_error ("%s", error->message);

services = dex_await_object (rebonjour_client_lookup_instances (client,
                                                                0,
                                                                "_ipp._tcp",
                                                                NULL,
                                                                REBONJOUR_LOOKUP_FLAGS_NONE),
                             &error);

The 0 there means any interface. Passing NULL for the domain uses local. The common case should not require looking up interface indexes which I’m pretty sure most people reading this have never even done before.

Advertising

Advertising is where things get more system-policy-oriented.

With Avahi, registration goes through Avahi’s D-Bus API. With systemd-resolved, registration uses RegisterService and UnregisterService, which are polkit-protected. Also, resolved needs full mDNS enabled with MulticastDNS=yes; MulticastDNS=resolve is enough to browse and resolve, but not enough to respond as a service.

So librebonjour can expose one API for registration, but it cannot make host policy disappear. Applications still need to handle authorization failure, missing mDNS responder support, sandbox boundaries, or whatever policy the system administrator has decided is appropriate.

That seems like the right way to demarcate things. The library should hide the provider mechanics, not the permissions of the platform.

Why

Mostly because I wanted this to exist.

DNS-SD is handy. Local-network service discovery is still useful. But using it from a GLib application means either caring too much about the provider or writing just enough glue that every application gets to have its own slightly different version of the same code.

And even worse is having to bundle things to build projects like Avahi for Flatpak when you only use the library which calls into D-Bus anyway.

This is not a grand platform initiative. It is not something I am employed to maintain. So you know, use wisely.

I’ve been putting together varlink-glib, which is a library for writing Varlink clients and services in C. The basic idea is to keep the transport policy out of the library. You get a connected GIOStream however you want, whether that is GLib networking, socket activation, or something more specialized, and then wrap it in a VarlinkClientProtocol or VarlinkServerProtocol.

The API is built around DexFuture, which makes the async parts feel a lot nicer in C than the usual callback layering. Client calls return a future, server replies return a future, and internally the protocol can use fibers for the work that wants to look sequential while still integrating with the GLib main context. This is very much the style of API I want for system services: explicit enough that you can debug it, but not so painfully manual that every call site becomes a state machine.

future = example_calc_add_call_invoke (proxy,
                                       call,
                                       VARLINK_METHOD_CALL_DEFAULT,
                                       add_reply_cb,
                                       NULL,
                                       NULL);

There is also a varlink-codegen tool which takes a .varlink interface and generates typed C wrappers for it. That gives you proxy objects, server skeletons, call objects, reply objects, and error constants instead of making every application hand-roll JSON. You can still drop down to VarlinkMessage, VarlinkMessageBuilder, and VarlinkMessageReader for forwarding or weird infrastructure cases, but most code should get to stay typed.

File descriptor passing works when the transport is a Unix socket connection. This follows the same general model as systemd’s Varlink support: the JSON payload contains an integer index, while the actual descriptors are sent out-of-band with SCM_RIGHTS and attached to the containing message as a GUnixFDList. Generated code can continue to treat the field as an integer, while the actual descriptor list stays attached to the underlying VarlinkMessage.

fd_list = g_unix_fd_list_new ();
fd_index = g_unix_fd_list_append (fd_list, fd, &error);

varlink_message_set_unix_fd_list (parameters, fd_list);

Protocol upgrades are supported too. A method call can ask to upgrade the connection, and once the final successful reply is sent, Varlink stops being valid on that connection. The VarlinkProtocol is still a GIOStream, so the next protocol can continue reading and writing through the same object. That keeps the handoff explicit without requiring a separate transport abstraction.

I also wired in optional Sysprof capture support. When enabled, client and server RPCs can show up as Sysprof marks with useful bits like method name, result, reply count, one-way, multi-reply, upgrade, Varlink error, and GError details. That matters because once you have concurrent calls, generated dispatch through VarlinkServerRegistry, and services doing real work, “it got slow somewhere” is not enough information.

There is still more polish to do, but the shape is there: typed generated APIs for normal users, low-level message APIs for infrastructure, DexFuture for async flow, Unix FD passing for the system service cases, protocol upgrades for handoff cases, and profiler hooks so it can be debugged when the happy path stops being happy.

Numbers

So what does that look like performance wise you might ask? For a very simple Echo interface in both D-Bus and Varlink you can get a rough estimate. No daemons, just serialization on a socketpair(). I haven’t started performance tuning yet so there may be ground to make up on both sides. But the answer is that the testcase for varlink-glib is about 5x faster than the testcase for GDBusConnection in either synchronous or asynchronous modes.

This doesn’t apply to all use-cases of D-Bus of course. But for a specific case I use it for (P2P IPC between peer processes), it is pretty big difference.

A small personal note: as I wrote in my recent update from France, I am no longer employed by Red Hat. Work like this is currently self-funded, out of pocket, while my family and I settle into a new chapter. If you find it useful, a note of encouragement or a contribution means a lot right now. It helps make it possible to keep improving the free software infrastructure many of us rely on.

I have been spending more time learning French lately, and as often happens, that turned into a small side project. liblingua is not intended to be a big mainstream translation platform. It is a fun GLib/GObject library for experimenting with local machine translation from applications.

The library uses Bergamot from Mozilla for the translation backend. Instead of sending text to a web service, liblingua resolves the language pair you ask for, downloads the required language model into the local user cache, and then performs translation locally.

The high-level API is built around a few small objects: LinguaRegistry discovers available translation profiles, LinguaProfile represents a model that can be loaded or downloaded, LinguaProgress reports download progress, and LinguaTranslator performs the translation.

All potentially blocking work is exposed as DexFuture, so it fits naturally into libdex based applications. If you are already using fibers, the code can stay linear and easy to read with dex_await_object().

A Small Example

Here is the basic shape of translating French into English:

#include <liblingua.h>

static void
translate_example (void)
{
  g_autoptr(LinguaProgress) progress = lingua_progress_new ();
  g_autoptr(LinguaRegistry) registry = NULL;
  g_autoptr(LinguaProfile) profile = NULL;
  g_autoptr(LinguaTranslator) translator = NULL;
  g_autoptr(LinguaTranslation) result = NULL;
  g_autoptr(GListModel) profiles = NULL;
  g_autoptr(GError) error = NULL;

  if ((registry = dex_await_object (lingua_registry_new (), &error)) &&
      (profiles = lingua_registry_resolve (registry, "fr", "en")) &&
      (profile = g_list_model_get_item (profiles, 0)) &&
      (translator = dex_await_object (lingua_profile_load (profile, progress), &error)) &&
      (result = dex_await_object (lingua_translator_translate (translator, "Bonjour"), &error)))
    g_print ("%s\n", lingua_translation_get_translation (result));
  else
    g_printerr ("Error: %s\n", error->message);
}

The first time a model is needed, loading the profile may download it. After that, the model is reused from the local cache. That makes liblingua useful for little tools, demos, and desktop experiments where local translation is preferable to wiring everything through a remote service.

In the future this is probably the type of thing we would want as a desktop service to avoid duplicating caches amongst Flatpak applications. It would also be extremely useful to do live translation in Camera and Image Preview apps. I played a bit with that using Tesseract for OCR and it worked better than expected.

Libdex now has DexLimiter, a small utility for bounding how much asynchronous work runs at once.

This is useful when a workload can produce more parallelism than the underlying machine, subsystem, or service should actually handle. Common examples include indexing files, downloading URLs, generating thumbnails, parsing documents, or querying a service with a fixed concurrency budget.

The usual API is dex_limiter_run(). It acquires a permit, starts a fiber, and releases the permit when that fiber finishes.

static DexFuture *
load_one_file (gpointer user_data)
{
  GFile *file = user_data;

  return dex_file_load_contents_bytes (file);
}

DexLimiter *limiter = dex_limiter_new (8);
DexFuture *future = dex_limiter_run (limiter,
                                     NULL,
                                     0,
                                     load_one_file,
                                     g_object_ref (file),
                                     g_object_unref);

In this example, no more than eight file loads will run at the same time, regardless of how many files are queued. The returned DexFuture resolves or rejects with the result of the spawned fiber.

One important detail is that dropping the returned future does not cancel a fiber that has already started. Once work has acquired a permit, it is allowed to complete so that the limiter can release the permit cleanly.

For more specialized cases, DexLimiter also supports manual acquire and release:

g_autoptr(GError) error = NULL;

if (dex_await (dex_limiter_acquire (limiter), &error))
  {
    do_limited_work ();
    dex_limiter_release (limiter);
  }

This is useful when the limited section is not naturally represented by a single fiber. However, callers must release exactly once for every successful acquire. In most cases, dex_limiter_run() is preferable because it handles release on both success and failure paths.

The limit should describe the constrained resource, not the number of items being processed. Remote APIs and databases may need a small limit. CPU-heavy work should usually be near the amount of useful worker parallelism. Local I/O can often tolerate a larger value, depending on the storage system. Separate resources should usually have separate limiters, so one workload does not consume another workload’s concurrency budget.

Finally, dex_limiter_close() can be used during shutdown. Once closed, pending and future acquisitions reject with DEX_ERROR_SEMAPHORE_CLOSED. Work that already holds a permit may continue, but releasing after close does not make new permits available.

The goal is to make bounded parallelism simple: queue as much asynchronous work as you need, but only run as much of it as the system should handle.

For about the past month, I have no longer been with Red Hat.

That is a strange sentence to write after so many years, but life has a way of changing the scenery whether or not one has finished packing. My family and I have made it safely to France, and we are quite happy here. The light is different, the pace is different, and there is a great deal to learn. For now, that is exactly where our attention needs to be.

I also think there is a broader lesson here for people whose safety, immigration status, or family stability may depend on employer flexibility. Do not assume that long tenure, remote work history, or prior verbal guidance will be enough. My own experience left me with the uncomfortable conclusion that these processes can become very narrow exactly when the human stakes are highest. Get things in writing, understand the policy surface area, and protect your family first.

This also means that some of the things I wrote about in my earlier mid-life update remain unresolved. I am currently in France on a visitor visa, which does not authorize work here. Our focus is on integration, language learning, and getting ourselves properly settled for the long term. That takes time, patience, paperwork, and a certain tolerance for being a beginner again.

As a result, I will not be taking on ongoing software maintenance responsibilities for the foreseeable future. I may still scratch the occasional itch where it directly improves my own computing life, but I am not currently able to provide the kind of broad, sustained stewardship that many projects deserve.

That is not an easy thing to say. Open source is entering an especially difficult period. We are now seeing AI systems used not only to generate code, but also to probe, disrupt, and attack critical software infrastructure. I suspect this will have a negative effect on the maintenance burden for a lot of projects, particularly the foundational pieces that distributions, companies, and users all rely on without always seeing the people behind them.

But there is a limit to what can reasonably be carried as unpaid labor, especially when the primary financial beneficiaries are downstream organizations with considerably more resources than the individual maintainers doing the work. At the moment, I need to prioritize my family, our stability, and the next chapter of our life here.

That said, I am still reachable for appropriate professional inquiries, advisory conversations, or consulting opportunities where the structure and location make sense. The best address for that now is christian at sourceandstack dot com.

For now, we are safe, settling in, and doing our best to build something durable out of a rather odd moment in the world. There are worse places to begin again than France.

The past few months have been heavy for many people in the United States, especially families navigating uncertainty about safety, stability, and belonging. My own mixed family has been working through some of those questions, and it has led us to make a significant change.

Over the course of last year, my request to relocate to France while remaining in my role moved up and down the management chain at Red Hat for months without resolution, ultimately ending in a denial. That process significantly delayed our plans despite providing clear evidence of the risks involved to our family. At the beginning of this year, my wife and I moved forward by applying for long-stay visitor visas for France, a status that does not include work authorization.

During our in-person visa appointment in Seattle, a shooting involving CBP occurred just a few parking spaces from where we normally park for medical outpatient visits back in Portland. It was covered by the news internationally and you may have read about it. Moments like that have a way of clarifying what matters and how urgently change can feel necessary.

Our visas were approved quickly, which we’re grateful for. We’ll be spending the next year in France, where my wife has other Tibetan family. I’m looking forward to immersing myself in the language and culture and to taking that responsibility seriously. Learning French in mid-life will be humbling, but I’m ready to give it my full focus.

This move also means a professional shift. I’ve dedicated a substantial portion of my time to maintaining and developing key components across the GNOME platform and its surrounding ecosystem. These projects are widely used, including in major Linux distributions and enterprise environments, and they depend on steady, ongoing care.

For the better part of two decades I’ve been putting in well beyond forty hours each week maintaining and advancing this stack. That level of unpaid or ad-hoc effort isn’t something I can sustain, and my direct involvement going forward will be very limited. Given how widely this software is used in commercial and enterprise environments, long-term stewardship really needs to be backed by funded, dedicated work rather than spare-time contributions.

If you or your organization depend on this software, now is a good time to get involved. Perhaps by contributing engineering time, supporting other maintainers, or helping fund long-term sustainability.

The folliwing is a short list of important modules where I’m roughly the sole active maintainer:

• GtkSourceView – foundation for editors across the GTK eco-system
• Text Editor – GNOME’s core text editor
• Ptyxis – Default terminal on Fedora, Debian, Ubuntu, RHEL/CentOS/Alma/Rocky and others
• libspelling – Necessary bridge between GTK and enchant2 for spellcheck
• Sysprof – Whole-systems profiler integrating Linux perf, Mesa, GTK, Pango, GLib, WebKit, Mutter, and other statistics collectors
• Builder – GNOME’s flagship IDE
• template-glib – Templating and small language runtime for a scriptable GObject Introspection syntax
• jsonrpc-glib – Provides JSONRPC communication with language servers
• libpeas – Plugin library providing C/C++/Rust, Lua, Python, and JavaScript integration
• libdex – Futures, Fibers, and io_uring integration
• GOM – Data object binding between GObject and SQLite
• Manuals – Documentation reader for our development platform
• Foundry – Basically Builder as a command-line program and shared library, used by Manuals and a future Builder (hopefully)
• d-spy – Introspect D-Bus connections
• libpanel – Provides IDE widgetry for complex GTK/libadwaita applications
• libmks – Qemu Mouse-Keyboard-Screen implementation with DMA-BUF integration for GTK

There are, of course, many other modules I contribute to, but these are the ones most in need of attention. I’m committed to making the transition as smooth as possible and am happy to help onboard new contributors or teams who want to step up.

My next chapter is about focusing on family and building stability in our lives.

Much like the s3-glib library I put together recently, I had another itch to scratch. What would it look like to have a PostgreSQL driver that used futures and fibers with libdex? This was something I wondered about more than a decade ago when writing the libmongoc network driver for 10gen (later MongoDB).

pgsql-glib is such a library which I made to wrap the venerable libpq PostgreSQL state-machine library. It does operations on fibers and awaits FD I/O to make something that feels synchronous even though it is not.

It also allows for something more “RAII-like” using g_autoptr() which interacts very nicely with fibers.

API Documentation can be found here.

My Red Hat colleague Jan Wildeboer made some posts recently playing around with Garage for S3 compatible storage. Intrigued by the prospect, I was curious what a nice libdex-based API would look like.

Many years ago I put together a scrappy aws-glib library to do something similar but after changing jobs I failed to find any free time to finish it. S3-GLib is an opportunity to do it better thanks to all the async/future/fiber support I put into libdex.

I should also mention what a pleasure it is to be able to quickly spin up a Garage instance to run the testsuite.

This week was Thanksgiving holiday in the US and I spent the entire week quite sick with something that was not Covid nor Flu according to rapid tests.

Managed to get a few things done through sheer force of will. I don’t recommend it.

Red Hat

• A lot of misdirection related to our international move towards France where my wife has family. The policies at Red Hat may mean that we need to come up with a strategy to keep a million-plus lines of code maintained if my only option to protect my family is to go the route of long-stay-visa (e.g. work not permitted).

  Some very lovely people have reached out and thank you for that!

  Touch base if you have options that would allow me to work from France if you like my approach to engineering. While I’d love to continue working on GNOME/GTK and related technologies, some things are more important and so I’m flexible.

  Lets all hope that the Corporate Leadership Team at Red Hat determines that supporting multi-ethnic families get safely out of the United States near their support network is of value to our mutual success.

Ptyxis

• Triage a handful of issues. This project has a pretty high-velocity file-rate while having an pretty low “bug exists here” rate. This legitimately burns a lot of my time every week that could be spent creating things.

  So if you are going to file issues in projects like Ptyxis or Text Editor, please take the time to do basic binary-search on where the problem really exists. Points given just for trying.

  In fact, this is a rant about how you should do that in all aspects of your life. Cut the problems you have in half — and half again.

Libdex

• Fumbled my way through adding a gi override to integrate DexFuture with asyncio. This allows for await some_future() but you need to make sure you have a GLib main context running as well and integrated with asyncio.

  Hopefully this means that you will soon be able to easily use all the nice Foundry APIs from Python with relative ease.

Foundry

• Add support for communicating with SSH agent for simple signing requests. Was surprised to not find this in libssh2 but maybe I missed something. We already link against that for libgit2 support.

• Add support for signing commits in the FoundryGitCommitBuilder commit helper. This supports GPG and SSH commit signing (no X509) because I have no need for it. Though the SSH form is tested best.

• Iteration on tracking changes to untracked/unstaged/staged files while using the commit builder API.

• Abstract commit signing into generic buffer signing so that it can be used for more things in the future (like signing tags).

• Add support for staging/unstaging files, hunks, and lines.

• New test-git-commit-builder-gtk to be able to test out the infrastructure for commit creation. It’s extremely scrappy but gets the job done enough to test the API out even if not very ergonomic.

• New foundry_git_vcs_stash() helper to be like git stash.

• Lots of new APIs around generating diffs, deltas, and patches.

• Lots of refactoring of Git subsystem to make the combination of threading and convenient APIs easier to maintain.

Builder

• Some work on git change management panel, diff viewer, etc

• VCS history panel which is useful as auxiliary for files as well as for viewing diffs.

• Details auxiliary panel for forge issues/merge-requests

• Experiment with using AdwPreferencesGroup and friends for creating AdwBottomSheet style menus in narrow mode (e.g. mobile).

One of my original hopes for Libdex was to help us structure complex asynchronous code in the GNOME platform. If my work creating Foundry and Sysprof are any indicator, it has done leaps and bounds for my productivity and quality in that regard.

Always in the back of my mind I hoped we could make those futures integrate with language runtimes.

This morning I finally got around to learning enough of Python’s asyncio module to write the extremely minimal glue code. Here is a commit that implements integration as a PyGObject introspection override. It will automatically load when you from gi.repository import Dex.

This only integrates with the asyncio side of things so your application is still responsible for integrating asyncio and GMainContext or else nothing will be pumping the DexScheduler. But that is application toolkit specific so I must leave that to you.

I would absolutely love it if someone could work on the same level of integration for GJS as I have even less experience with that platform.