Happenings in GNOME

At Red Hat we are expected to set, and meet, goals each quarter if we want our “full” bonus. One of those is around introducing AI into our daily work. You’ve probably seen various Red Hat employees talking about using AI. It’s hard to deny there is financial incentive to do so.

Astute students of behavioral science know that humans work harder to not lose something than to gain something new. Arguably it’s only a “mandate to use AI” if you are entitled to the revenue so attaching it to your bonus is a convenient way to both “not be a mandate” and take advantage of the human behavior to not lose something.

La mujer del César no solo debe ser honesta, sino parecerlo.

Conveniently, I got pretty fed up with Debug Adapter Protocol (DAP for short) this week. So I wondered how hard it would be to just talk the MI2 protocol like Builder does (using the GPLv3+ gdbwire) but with a libdex-oriented library. It could potentially make integrating GDB into Foundry much easier.

Red Hat has so generously given us “ephemeral credits” at Cursor.com to play with so I fired up the cursor-agent command line tool and got started.

My goal here is just to make a minimal API that speaks the protocol and provides that as DexFuture with a very shallow object model. The result for that is mi2-glib. This post is mostly about how to go about getting positive results from the bias machine.

Writing a fully featured library in a couple hours is certainly impressive, regardless of how you feel about AI’s contributions to it.

Before I continue, I want to mention that this is not used in Foundry and the GDB support which landed today is using DAP. This was just an experiment to explore alternatives.

Create Project Scaffolding

To get the project started I used Foundry’s shared library template. It creates a C-based project with pkg-config, GObject Introspection support, Vala *.vapi generation, versioning with ABI mechanics, and gi-doc based documentation. Basically all the tricky things just work out of the box.

You could probably teach the agent to call this command but I choose to do it manually.

foundry template create library
Location[/home/christian]: .

The name for your project which should not contain spaces
Project Name: mi2-glib

The namespace for the library such as "Mylib"
Namespace: Mi2
 1: No License
 2: AGPL 3.0 or later
 3: Apache 2.0
 4: EUPL 1.2
 5: GPL 2.0 or later
 6: GPL 3.0 or later
 7: LGPL 2.1 or later
 8: LGPL 3.0 or later
 9: MIT
10: MPL 2.0
License[7]:
Version Control[yes]: 
mi2-glib/meson.options
mi2-glib/meson.build
mi2-glib/lib/meson.build
mi2-glib/testsuite/meson.build
mi2-glib/lib/mi2-glib.h
mi2-glib/lib/mi2-version.h.in
mi2-glib/lib/mi2-version-macros.h
mi2-glib/testsuite/test-mi2-glib.c
mi2-glib/doc/mi2-glib.toml.in
mi2-glib/doc/urlmap.js
mi2-glib/doc/overview.md
mi2-glib/doc/meson.build
mi2-glib/README.md
mi2-glib/LICENSE
mi2-glib/.foundry/.gitignore
mi2-glib/.foundry/project/settings.keyfile
mi2-glib/.git/objects
mi2-glib/.git/refs/heads
mi2-glib/.git/HEAD

Building Context

After that I added an AGENTS.md file that described how I want code to be written. I gave it some examples of my C style (basically opinionated GTK styling). That is things like preferring autoptr over manual memory management, how to build, how to test, etc. Consider this the “taste-making” phase.

The next thing I needed to do was to ensure that it has enough examples to be able to write libdex-based GObject code. So I copied over the markdown docs from libdex and some headers. Just enough for the agent to scan and find examples to prime how it should be used. Update your AGENTS.md to note where that documentation can be found and what it can expect to find there. That way the agent knows to look for more details.

Just as a matter of practicality, it would be great if we could have gi-doc generate markdown-formatted descriptions of APIs rather than just HTML.

Now that is all good and fun, but this is about mi2 now isn’t it? So we need to teach the agent about what mi2 is and how it works. Only then will it be able to write something useful for us.

GDB conveniently has a single-file version of the documentation so we need not piss off any network admins with recursive wget.

That left me with a big HTML document with lots of things that don’t matter. I wanted something more compact that will better fit into the context window of the agent. So I asked the agent to look over the HTML file and amalgamate a single gdb.md containing the documentation in a semi-structured format with markdown. That is of course still a lot of information.

Next, I went another level deeper where I asked it to extract from gdb.md information about the mi2 protocol. Any command in the mi2 protocol or related command in GDB is important and therefore should be included in a new mi2.md document. This will serve as the basis of our contextual knowledge.

Knowing the Problem Domain

Using an agent is no replacement for knowing the problem domain. Thankfully, I’ve written dozens of socket clients/servers using GIO primitives. So I can follow my normal technique.

First we create a client class which will manage our GIOStream. That may be a GSocketConnection or perhaps even a GSimpleIOStream containing memory streams or file streams to enable unit testing.

Then we move on to a GDataInputStream which can read the format. We ask the agent to subclass appropriately and provide a DexFuture based wrapper which can read the next message. It should have enough context at this point to know what needs to be read. Start simple, just reading the next message as a bag of bytes. We parse it into structured messages later.

After that do the same for a GDataOutputStream subclass. Once we have that we can ask the agent to wrap the input/output streams inside the client. All pretty easy stuff for it to get right.

Where I found it much easier to write things myself was dealing with the mechanics of a read-loop that will complete in-flight operations. Small price to pay to avoid debugging the slop machine.

After these mechanics are in place we can start on the message type. I knew from my previous debugger work in Builder that there would be a recursive result type and a toplevel container for them. I decided to name these Message and Result and then let the agent generate them. Even the first try had something “usable” even if I would say not very tasteful. It kept trying to do manual memory management. It left lots of “TODO” in place rather than actually doing the thing.

Instead of trying to describe to the agent how its implementation was ugly and incomplete I realized my time would be better spent letting it figure it out. So I shifted my focus towards a test suite.

Setting up a Test Suite

I saw the biggest improvement from the agent after I setup a test suite. This allowed the agent to run ninja test to check the results. That way it wasn’t just about satisfying GCC compiler warnings but also that it was functional.

I would start this as early as you can because it pays dividends. Make sure you provide both good and bad inputs to help guide it towards a better implementation.

The biggest win was creating tests for the input stream. Create different files with good and bad input. Since they wrap a base stream it is pretty easy to mock that up with a GFileInputStream or GMemoryOutputStream and verify the results. If the tests provide enough context through debug messages the agent can feed that back into why the error occurred. This made the iteration loop remarkably fast.

Another extremely valuable thing to do is to setup some type of sanitizer in your build configuration. I often build with meson configure -Db_sanitize=address to help catch memory errors. But also do things like assert object finalization in your testsuite to make sure that leaks don’t hide them.

Anyway, writing this post took about as long as designing the core parts of the library and that, I guess in some small way, is pretty neat.

It’s the time of year where the Oregon allergens has me laid out. Managed to get some stuff done while cranking up the air purifier.

VTE

• Work on GtkAccessibleHypertext implementation. Somewhat complicated to track persistent accessible objects for the hyperlinks but it seems like I have a direction to move forward.

Ptyxis

• Fix CSS causing too much padding in header bar 49.0

• Make --working-directory work properly with --new-window when you want you also are restoring a session.

• Add support for cell-width control in preferences

• Move an issue to mutter for more triage. Super+key doesn’t get delivered to the app until it’s pressed a second time. Not sure if this is by design or not.

Foundry

• Merge support for peel template in GTK/Adwaita

• Lots of work on DAP support for FoundryDebugger

• Some improvements to FOUNDRY_JSON_OBJECT_PARSE and related helper macros.

• Handle failure to access session bus gracefully with flatpak build pipeline integration.

• Helper to track changes to .git repository with file monitors.

• Improve new SARIF support so we can get GCC diagnostics using their new GCC 16.0 feature set (still to land).

• Lots of iteration on debugger API found by actually implementing the debugger support for GDB this time using DAP instead of MI2.

• Blog post on how to write live-updating directory list models.

• Merge FoundryAdw which will contain our IDE abstractions on top of libpanel to make it easy to create Builder like tooling.

• Revive EggLine so I have a simple readline helper to implement a testing tool around the debugger API.

• Rearrange some API on debuggers

• Prototype a LLDB implementation to go along with GDB. Looks like it doesn’t implement thread events which is pretty annoying.

• Fix issue with podman plugin over-zealously detecting it’s own work.

Builder

• Make help show up in active workspace instead of custom window so that it can be moved around.

Libdex

• Write some blog posts

• Add an async g_unlink() wrapper

• Add g_find_program_in_path() helper

Manuals

• Fix app-id when using devel builds

• Merge support for new-window GAction

Libpanel

• Merge some improvements to the changes dialog

Text Editor

• Numerous bugs which boil down to “ibus not installed”

D-Spy

• Issue tracker chatter.

I took a different approach to directory listings in Foundry. They use GListModel as the interface but behind the scenes it is implemented with futures and and fibers.

A primary use case for a directory listing is the project tree of an IDE. Since we use GtkListView for efficient trees in GTK it we expose a GListModel. Each item in the directory is represented as a FoundryDirectoryItem which acts just a bit like a specialized GFileInfo. It contains information about all the attributes requested in the directory listing.

You can also request some other information that is not traditionally available via GFileInfo. You can request attributes that will be populated by the version control system such as if the file is modified or should be ignored.

Use a GtkFilterListModel to look at specific properties or attributes. Sort them quickly using GtkSortListModel. All of this makes implementing a project tree browser very straight forward.

Reining in the Complexity

One of the most complex things in writing a directory listing is managing updates to the directory. To manage some level of correctness here Foundry does it with a fiber in the following ordering:

• Start a file monitor on the directory and start queing up changes
• Enumerate children in the directory and add to internal list
• Start processing monitor events, starting with the backlog

Performance

There are a couple tricks to balancing the performance of new items being added and items being removed. We want both to be similarly well performing.

To do this, new items are always placed at the end of the list. We don’t care about sorting here because that will be dealt with at a higher layer (in the GtkSortListModel). That keeps adding new items quite fast because we can quickly access the end of the list. This saves us a lot of time sorting the tree just for the removal.

But if you aren’t sorting the items, how do you make removals quick? Doesn’t that become O(n)?

Well not quite. If we keep a secondary index (in this case a simple GHashTable) then we can store a key (the file’s name) which points to a stable pointer (a GSequenceIter). That lookup is O(1) and the removal from a GSequence on average are O(log n).

Big O notation is often meaningless when you’re talking about different systems. So let’s be real for a moment. Your callback from GListModel::items-changed() can have a huge impact on performance compared to the internal data structures here.

Reference Counting and Fibers

When doing this with a fiber attention must be taken to avoid over referencing the object or you risk never disposing/finalizing.

One way out of that mess is to use a GWeakRef and only request the object when it is truly necessary. That way, you can make your fiber cancel when the directory listing is disposed. In turn your monitor and other resources are automatically cleaned up.

For example:

GWeakRef *wr = g_new0 (GWeakRef, 1);
g_weak_ref_init (wr, self);
self->fiber = dex_scheduler_spawn (NULL, 0, fiber_func,
                                   wr, free_weak_ref);

And in dispose you can simply dex_clear (&self->fiber); and the fiber will automatically cancel.

In the fiber I start by creating a set of futures that will resolve when there is more work to be done. Then I release my self object followed by awaiting the future. At that point I’ll either resolve or reject from an error (including the fiber being cancelled).

If I need self again because I got an event, it’s just a g_weak_ref_get() away.

Future-based File Monitors

I’m not a huge fan of “signal based” APIs like GFileMonitor so internally FoundryFileMonitor does something a bit different. It still uses GFileMonitor internally for outstanding portability, but the exposed API uses DexFuture.

This allows you to call when you want to get the next event (which may already be queued). Call foundry_file_monotor_next() to get the next event and the future will resolve when an event has been received. This makes more complex awaiting operations feasible too (such as monitoring multiple directories for the next change).

foundry-directory-listing.c, foundry-directory-item.c, and foundry-file-monitor.c.

In addition to all the Libdex 1.0 fanfare, Foundry has also reached 1.0 for GNOME 49. That doesn’t mean it’s complete, but it does mean that what is there I feel pretty confident about from an API/ABI standpoint.

If you have a project that works in GNOME Builder, it is a good time to test it out with Foundry! To get started you need to “initialize” a Foundry project similar to what you do with Git. This creates a minimal skeleton structure that Foundry will use for the project.

cd myapp/
foundry init

At this point, you should have a .foundry/ directory in your project. Foundry will try to store everything it knows about your project in there. That includes settings, builds, cache, and tmp files. It also sets up a .gitignore file so that only the things you’re interested end up getting committed to the project’s git repository.

Now we can build the project.

foundry build

Foundry will do all the same sort of SDK setup, cross-container build pipelines, and build system management that GNOME Builder does.

To run the project, just run the following. It will handle building first if necessary too.

foundry run

If using a Flatpak manifest as your build configuration (you can check with foundry config list) then it will handle all the same sort of details that Builder does.

Building with Libfoundry

Of course the command line tool is great for when you are needing to quickly do something in a terminal. But perhaps you want to integrate Foundry features into your own application or tools? Everything in the command line tool is available via libfoundry.

To open an existing project with libfoundry, using just a directory to start, we can discover the .foundry directory location.

g_autofree char *state_dir = NULL;
DexFuture *future;

future = foundry_context_discover (g_get_current_dir (), NULL);
state_dir = dex_await_string (future, &error);

We can use that information to load the project.

g_autoptr(FoundryContext) context = NULL;
DexFuture *future;

future = foundry_context_new (state_dir, NULL, 0, NULL);
context = dex_await_object (future, &error);

To run a build we need to get the build manager service. That is available from the context with the :build-manager property.

g_autoptr(FoundryBuildManager) build_manager = NULL;

build_manager = foundry_context_dup_build_manager (context);

One thing you’ll notice in Foundry is that many getters take a reference to the returned value. This is because Foundry heavily uses fibers and it is always better to own your objects when you cross fiber suspension points. This is where the dup_ prefix comes from instead of get_.

Building the project is quite simple too. Just await until it has completed.

dex_await (foundry_build_manager_build (build_manager), &error);

Running with Libfoundry

To run is quite similar. Except you use the run manager instead of the build manager.

g_autoptr(FoundryRunManager) run_manager = NULL;

run_manager = foundry_context_dup_run_manager (context);
dex_await (foundry_run_manager_run (run_manager), &error));

There are of course more detailed ways to do this if you need precise control over things like where the project is deployed, such as to an external device. Or perhaps you want to control what command is run. The run manager and build manager also allows controlling the PTY that is used for both operations.

Hopefully that is just enough to get you excited about using the tooling. The API has been obsessed over but it could use more documentation of which this is the infancy.

I’ve talked a lot about fibers before. But what are they and how do they work?

From the programmer perspective they can feel a lot like a thread. They have a stack just like a real thread. They maintain a program counter just like a real thread. They can spill registers to the stack like a real thread.

Most importantly is that you can call into GTK from your fiber if running on the main thread. Your fiber will have been dispatched from the main loop of the GTK thread so it’s safe to do.

There are downsides too though. You have to allocate stack and guard pages for them like a real thread. They have some cost in transitioning between stacks even if fairly low these days. You also need to be mindful to own the lifecycle of pointers on your stack if you intend to “await” (e.g. suspend your fiber).

Many fibers may work together on a single thread where each runs a little bit until yielding back to the scheduler. This is called “cooperative multi-tasking” because it is up to the fibers to be cooperative and yield when appropriate.

That means that you generally should not “block” when writing code for fibers. It not only blocks your own fiber from making progress but all other fibers sharing your thread.

The way around this is to use non-blocking APIs instead of blocking calls like open() or read(). This is where combining a library for Futures and a library for Fibers makes a lot of sense. If you provide asynchronous APIs based on futures you immediately gain a natural point to yield from a fiber back to the scheduler.

The scheduler maintains two queues for fibers on a thread. That is because fibers exist in one of two (well three sort of) states.

• The first state is “runnable” meaning the fiber can make progress immediately.
• The second state is “blocked” meaning the fiber is waiting on a future to complete.
• The (sort of) third state is “finished” but in libdex it would be removed from all queues here.

When a fiber transitions from runnable to blocked its linked-list node migrates from one queue to the other. Naturally, that means we must be waiting for a completion of a future. The scheduler will register itself with the dependent future so it may be notified of completion.

Upon completion of the dependent future our fiber will move from the blocked to the runnable queue. The next GMainContext iteration will transition into the fibers stack, restore register values, set the instruction pointer and continue running.

Fibers get their stack from a pool of pre-allocated stacks. When they are discarded they return to the pool for quick re-use. If we have too many saved then we release the stacks memory back to the system. It’s all just mmap()-based stacks currently.

You might be wondering how we transition into the fibers stack from the thread. Libdex has a few different strategies for that based on the platforms it supports.

Windows, for example, has native support for fibers in their C runtime. So it uses ConvertThreadToFiber() and ConvertFiberToThread() for transitions.

On Linux and many other Unix-like systems we can use makecontext() and swapcontext() to transition. There was a time when swapcontext() was quite slow and so people used specialized assembly to do the same thing. These days I found that to be unnecessary (at least on Linux).

Another way to transition stacks is by using signalstack(), but libdex does not use that method.

Libdex fibers work on Linux, FreeBSD, macOS (both x86_64 and aarch64), Windows, and Illumos/Solaris. I believe Hurd also works but I’ve not verified that.

When your fiber first runs you will be placed at the base of your new stack. So if you found yourself in a debugger at your fibers entry point, it might look like you are one function deep. The first function from libdex would be equivalent to your _start on a regular thread.

If you await while 5 functions deep then your stacks register state will be saved and then your stack is set aside. The fiber will transition back to the scheduler where it left off. Then the original thread state is restored and the fiber scheduler can continue on to the next work item.

At the core, the fiber scheduler is really just a GSource within the GMainContext. It knows when it can flag itself is runnable. When dispatched it will wake up any number of runnable fibers.

To make sure that we don’t have to deal with extremely tricky situations fibers may not be migrated across threads. They are always pinned to the thread they were created on. If that becomes a problem it is usually better to break up your work into smaller tasks.

Another feature that has become handy is implicit fiber cancellation. A fiber is itself a future. If all code awaiting completion of your fiber have discarded interest then your fiber will be implicity cancelled.

Where this works out much better than real thread cancellation is that we already have natural exit points where we yield. So when your fiber goes to dex_await() it will get a DEX_ERROR_FIBER_CANCELLED in a GError. Usually when you get errors you propagate that rejection by returning from your fiber, easy.

If you do not want implicit fiber cancellation, you can “disown” your fiber using dex_future_disown().

In the future I’d love to approach stackless coroutines but that will be a taller order without compiler help or introducing complexity through macro madness.

VTE

• Little back-and-forth on what we can do to improve a11y further with VTE. Prototype’d a way to extract hyperlinks and provide them into AccessibleText, however that is not the right place.

  We really need an implementation of the at-spi hyperlink API in GTK so that VteTerminal may implement it.

• Merged a number of a11y fixes provided by Lukáš Tyrychtr which fix some of my original a11y implementation to follow the at-spi expectations better.

  Also added one of my own to fix a potential uint underrun when describing runs thanks to long to uint conversion.

Ptyxis

• Got a report of a bit of undesirable behavior when closing a window where close-able tabs are closed before showing the the special close dialog.

  It’s all more complex than I wish it was, because there are multiple ways we can enter that flow. Additionally, the asynchronous nature of closing tabs makes that state a bit “split brain” as it exists now. We may want to implement a “session” object to manage this stuff all in one place going forward.

• When showing multiple tabs the “visual bell” background did not cover the entire headerbar. Quick CSS fix that.

• Sebastian Wick needed a terminal with the Terminal Intents implemented so I took a crack at implementing that in Ptyxis. That allowed forward-progress on implementing the GLib side of things.

  Doing so found a couple of things we want to address in the upcoming FreeDesktop terminal intent “standard” and thankfully they’ve been moving that forward (including the Ptyxis-side).

Libpeas

• libpeas distro packaging was failing on the testsuite for Lua due to LGI not supporting girepository-2.0. Libpeas 2.x doesn’t use girepository at runtime, so this is relegated to the testsuite.

  For now, I’ve just disabled the Lua tests since they can’t possibly work due to that girepository usage.

  Some months back I provided patches for LGI to support girepository-2.0 but the project seems unmaintained and therefore hard for anyone to really say “yes merge this”.

  You can find some more information from Victoria Lacroix at https://www.vtrlx.ca/posts/2025/lgi-fork/

Builder

• They don’t get much testing during development so we had a small build regression on i686.

• Spent a bunch of time on what will hopefully become Builder 50. This is a major rewrite on top of Foundry, Foundry-Gtk, and what will become Foundry-Adw in the 1.1 release of Foundry.

  It is hard to describe how remarkably small this will make Builder as a project compared to previous versions. So much complexity in the Builder code-base was in response to all the async/finish flows required to keep things smooth. But now that we have libdex and libfoundry, it’s just so easy to implement that sort of magic.

Libdex

• Sebastian Wick had some great ideas on integrating libdex with gdbus-codegen. Discussed a few different ways we could go forward to make something ergonomic. It would be nice to implement both consuming as proxies and providing as skeletons with futures.

Libpanel

• I noticed that the bottom corner buttons in Builder were not aligning with the external rounded corners of the window. That was fallout from a previous cleanup so fixed that up before release.

Template-GLib

• Thanks to early testing by distributors we found an issue in the testsuite which was exercising GObject Introspection integration. It broke on 32-bit because the gsize parameters are different and I never implemented auto-casting of integers of different sizes.

  So if you did a call like GLib.utf8_substring(str, i64(3), i64(-1)) and were on 32-bit that would fail since the offset/length parameters are glong which is sized differently.

  The quick fix that will still allow for errors to propagate was to implement auto up/down casting of numbers so long as they will fit in the destination type. Otherwise, you’ll fail the cast operation and errors propagate as necessary.

  This fix landed shortly before 3.38.0 but was tested on numerous architectures before release.

Foundry

• Learned that even command-line tools get appdata now days so I went ahead and implemented that for the foundry CLI tool.

• We now dynamically link foundry binary. Originally I had plans to statically link it so that we can send it to a remote system and run it as a bridge. Since we aren’t there yet anyway, it doesn’t make sense to enforce that and/or make distributions patch it out.

• FoundryBuildManager got a :busy property which makes it much easier to implement UI like Builder has where you show a build button or a stop button based on status.

  It also got a new stop() method/GAction for that too. This is a lot harder than it looks because you need to plumb through a cancellable to all of the build pipeline API which is going to be awaiting the first future of [cancellable, some_op].

  Was extremely happy to see it work on the first try which means I’ve done something right in the libdex design.

• Once I did the above, adding a rebuild action and method was quite easy. We have all the necessary plumbing to either call an action or await a method future and get the same result.

• A last minute API was added to create a producer from a consumer PTY fd. Hopefully using this new nomenclature is not so confusing for people used to ancient PTY terminology which I know is extremely confusing to begin with. But we gotta move past those antiquated terms (which I wont repeat here) as they are both morally wrong and technically inaccurate.

• The FoundryFileManager can now automatically discover content-type when provided a filename. This vastly simplifies API usage when you have one-or-the-other to get a symbolic icon. Since we override many icons from the system, that is just a necessary abstraction.

• FoundrySearchResult got an :icon property which means it’s basically usable to search UI now. Though there are not many FoundrySearchProviders yet as they will land for 1.1.

• A new context.file-manager-show GAction is provided which allows you to pass a uri as a "s"-typed GVariant. I feel stupid for not doing this a decade ago in Builder and quite frankly, it should probably just exist in GTK.

• Libpanel/Builder has this nice “Action Muxer” API for years now and that is exported in Foundry too so we can use it in libfoundry-gtk and libfoundry-adw. It’s extremely handy when the built-in action support in GTK is not enough.

• Foundry has a SQLite database of extended attributes on URIs when the underlying file-system does not support extended attributes. I made a little boo-boo there so made that to actually work.

• Talked to dmalcom about supporting SARIF in Foundry for GNOME 50. They (the GCC project) would really like to see more consumers of it and Foundry is an obvious place to put that.

  It doesn’t look terribly difficult and it should allow us to drop the whole “regex parsing of PTY content” if done right.

Foundry-Gtk

• Fixed palette parsing and color application to VteTerminal.

• Imported all the Ptyxis terminal palettes which can now be represented as a FoundryTerminalPaletteSet and FoundryTerminalPalette.

  Ultimately, what this means is if you link against libfoundry and libfoundry-gtk you could functionally create your own Ptyxis replacement in very little code (assuming you don’t use an agent on the host OS like Ptyxis does).

  Just use the FoundrySdk as your container abstraction (which you can query using FoundrySdkManager) and FoundryTerminal with FoundryTerminalPaletteSet.

• You can now list/find available palettes with a GListModel API in the form of foundry_terminal_list_palette_sets() and foundry_terminal_find_palette_set(). Each set has an easy light/dark property you can use based on your needs.

• The file-search plugin now properly implements the load() vfunc for search results so you can implement preview/opening in apps.

Foundry-Adw

• A ton of work on the workspace, page, and panel APIs for 1.1. I’m really trying to find a way to re-use this across a number of applications such as Builder, Drafting, Sysprof, and more.

Releases

• gnome-text-editor 49.0

• gtksourceview 5.18.0

• sysprof 49.0

• gnome-builder 49.0

• ptyxis 49.0

• libdex 1.0.0

• foundry 1.0.0

• libpanel 1.10.2

• template-glib 3.38.0

• d-spy 49.0 (thanks to Jordan for the 49.1 w/ CI fixed)

• gom 0.5.4

• libpeas 2.2.0

• manuals 49.0

Other

• Team coffee hour, shared some battle wounds of trying to use AI for a pahole wrapper which fixed all my class structs to be cacheline aligned. Was just faster to write the damn code.

There are often needs to integrate with blocking APIs that do not fit well into the async or future-based models of the GNOME ecosystem. In those cases, you may want to use a dedicated thread for blocking calls so that you do not disrupt main loops, timeouts, or fiber scheduling.

This is ideal when doing things like interacting with libflatpak or even libgit2.

Creating a Dedicated Thread

Use the dex_thread_spawn() function to spawn a new thread. When the thread completes the resulting future will either resolve or reject.

typedef DexFuture *(*DexThreadFunc) (gpointer user_data);

DexFuture *future = dex_thread_spawn ("[my-thredad]", thread_func, thread_data,
                                      (GDestroyNotify)thread_data_free);

Waiting for Future Completion

Since dedicated threads do not have a Dex.Scheduler on them and are not a fiber, you may not await futures. Awaiting would suspend a fiber stack but there is no such fiber to suspend.

To make integration easier, you may use dex_thread_wait_for() to wait for a future to complete. The mechanism used in this case is a mutex and condition variable which will be signaled when the dependent future completes.

Previously, previously, and previously.

The Gio.IOStream APIs already provide robust support for asynchronous IO. The common API allows for different types of implementation based on the stream implementation.

Libdex provides wrappers for various APIs. Coverage is not complete but we do expect additional APIs to be covered in future releases.

File Management

See dex_file_copy() for copying files.

See dex_file_delete() for deleting files.

See dex_file_move() for moving files.

File Attributes

See dex_file_query_info() and dex_file_query_file_type(), and dex_file_query_exists() for basic querying.

You can set file attributes using dex_file_set_attributes().

Directories

You can create a directory or hierarchy of directories using dex_file_make_directory() and dex_file_make_directory_with_parents() respectively.

Enumerating Files

You can create a file enumerator for a directory using dex_file_enumerate_children().

You can also asynchronously enumerate the files of that directory using dex_file_enumerator_next_files() which will resolve to a g_autolist(GFileInfo) of infos.

Reading and Writing Files

The dex_file_read() will provide a Gio.FileInputStream which can be read from.

A simpler interface to get the bytes of a file is provided via dex_file_load_contents_bytes().

The dex_file_replace() will replace a file on disk providing a Gio.FileOutputStream to write to. The dex_file_replace_contents_bytes() provides a simplified API for this when the content is readily available.

Reading Streams

See dex_input_stream_read(), dex_input_stream_read_bytes(), dex_input_stream_skip(), and dex_input_stream_close() for working with input streams asynchronously.

Writing Streams

See dex_output_stream_write(), dex_output_stream_write_bytes(), dex_output_stream_splice(), and dex_output_stream_close() for writing to streams asynchronously.

Sockets

The dex_socket_listener_accept(), dex_socket_client_connect(), and dex_resolver_lookup_by_name() may be helpful when writing socket servers and clients.

D-Bus

Light integration exists for D-Bus to perform asychronous method calls.

See dex_dbus_connection_call(), dex_dbus_connection_call_with_unix_fd_list(), dex_dbus_connection_send_message_with_reply() and dex_dbus_connection_close().

We expect additional support for D-Bus to come at a later time.

Subprocesses

You can await completion of a subprocess using dex_subprocess_wait_check().

Foundry uses some helpers to do UTF-8 communication but I’d like to bring that to libdex in an improved way post-1.0.

Asynchronous IO with File Descriptors

Gio.IOStream and related APIs provides much opertunity for streams to be used asynchronously. There may be cases where you want similar behavior with traditional file-descriptors.

Libdex provides a set of AIO-like functions for traditional file-descriptors which may be backed with more efficient mechanisms.

Gio.IOStream typically uses a thread pool of blocking IO operations on Linux and other operating systems because that was the fastest method when the APIs were created. However, on some operating systems such as Linux, faster methods finally exist.

On Linux, io_uring can be used for asynchronous IO and is provided in the form of a Dex.Future.

Asynchronous Reads

DexFuture *dex_aio_read  (DexAioContext *aio_context,
                          int            fd,
                          gpointer       buffer,
                          gsize          count,
                          goffset        offset);

Use the dex_aio_read() function to read from a file-descriptor. The result will be a future that resolves to a gint64 containing the number of bytes read.

If there was a failure, the future will reject using the appropriate error code.

Your buffer must stay alive for the duration of the asynchronous read. One easy way to make that happen is to wrap the resulting future in a dex_future_then() which stores the buffer as user_data and releases it when finished.

If you are doing buffer pooling, more effort may be required.

Asynchronous Writes

DexFuture *dex_aio_write (DexAioContext *aio_context,
                          int            fd,
                          gconstpointer  buffer,
                          gsize          count,
                          goffset        offset);

A similar API exists as dex_aio_read() but for writing. It too will resolve to a gint64 containing the number of bytes written.

buffer must be kept alive for the duration of the call and it is the callers responsibility to do so.

You can find this article in the documentation under Asynchronous IO.

Previously, and previously.

Schedulers

The Dex.Scheduler is responsible for running work items on a thread. This is performed by integrating with the threads GMainContext. The main thread of your application will have a Dex.MainScheduler as the assigned scheduler.

The scheduler manages callbacks such as work items created with Dex.Scheduler.push(). This can include blocks, fibers, and application provided work.

You can get the default scheduler for the application’s main thread using Dex.Scheduler.get_default(). The current thread’s scheduler can be retrieved with Dex.Scheduler.ref_thread_default().

Thread Pool Scheduling

Libdex manages a thread pool which may be retrieved using Dex.ThreadPoolScheduler.get_default().

The thread pool scheduler will manage a number of threads that is deemed useful based on the number of CPU available. When io_uring is used, it will also restrict the number of workers to the number of uring available.

Work items created from outside of the thread pool are placed into a global queue. Thread pool workers will take items from the global queue when they have no more items to process.

To avoid “thundering herd” situations often caused by global queues and thread pools a pollable semaphore is used. On Linux, specifically, io_uring and eventfd combined with EFD_SEMAPHORE allow waking up a single worker when a work item is queued.

All thread pool workers have a local Dex.Scheduler so use of timeouts and other GSource features continue to work.

If you need to interact with long-blocking API calls it is better to use Dex.thread_spawn() rather than a thread pool thread.

Thread pool workers use a work-stealing wait-free queue which allows the worker to push work items onto one side of the queue quickly. Doing so also helps improve cacheline effectiveness.

Fibers

Fibers are a type of stackfull co-routine. A new stack is created and a trampoline is performed onto the stack from the current thread.

Use Dex.Scheduler.spawn() to create a new fiber.

When a fiber calls one of the Dex.Future.await() functions or when it returns the fiber is suspended and execution returns to the scheduler.

By default, fibers have a 128-kb stack with a guard page at the end. Fiber stacks are pooled so that they may be reused during heavy use.

Fibers are a Dex.Future which means you can await the completion of a fiber just like any other future.

Note that fibers are pinned to a scheduler. They will not be migrated between schedulers even when a thread pool is in use.

Fiber Cancellation

Fibers may be cancelled if the fiber has been discarded by all futures awaiting completion. Fibers will always exit through a natural exit point such as a pending “await”. All attempts to await will reject with error once a fiber has been cancelled.

If you want to ignore cancellation of fibers, use Dex.Future.disown() on the fiber after creation.

This article can be found at scheduling in the libdex documentation.

Previously.

Historically if you wanted to do asynchronous work in GObject-based applications you would use GAsyncReadyCallback and GAsyncResult.

There are two ways to integrate with this form of asynchronous API.

In one direction, you can consume this historical API and provide the result as a DexFuture. In the other direction, you can provide this API in your application or library but implement it behind the scenes with DexFuture.

Converting GAsyncResult to Futures

A typical case to integrate, at least initially, is to extract the result of a GAsyncResult and propagate it to a future.

One way to do that is with a Dex.Promise which will resolve or reject from your async callback.

static void
my_callback (GObject      *object,
             GAsyncResult *result,
             gpointer      user_data)
{
  g_autoptr(DexPromise) promise = user_data;
  g_autoptr(GError) error = NULL;

  if (thing_finish (THING (object), result, &error))
    dex_promise_resolve_boolean (promise, TRUE);
  else
    dex_promise_reject (promise, g_steal_pointer (&error));
}

DexFuture *
my_wrapper (Thing *thing)
{
  DexPromise *promise = dex_promise_new_cancellable ();

  thing_async (thing,
               dex_promise_get_cancellable (promise),
               my_callback,
               dex_ref (promise));

  return DEX_FUTURE (promise);
}

Implementing AsyncResult with Futures

In some cases you may not want to “leak” into your API that you are using DexFuture. For example, you may want to only expose a traditional GIO API or maybe even clean up legacy code.

For these cases use Dex.AsyncResult. It is designed to feel familiar to those that have used GTask.

Dex.AsyncResult.new() allows taking the typical cancellable, callback, and user_data parameters similar to GTask.

Then call Dex.AsyncResult.await() providing the future that will resolve or reject with error. One completed, the users provided callback will be executed within the active scheduler at time of creation.

From your finish function, call the appropriate propgate API such as Dex.AsyncResult.propagate_int().

void
thing_async (Thing               *thing,
             GCancellable        *cancellable,
             GAsyncReadyCallback  callback,
             gpointer             user_data)
{
  g_autoptr(DexAsyncResult) result = NULL;

  result = dex_async_result_new (thing, cancellable, callback, user_data);
  dex_async_result_await (result, dex_future_new_true ());
}

gboolean
thing_finish (Thing         *thing,
              GAsyncResult  *result,
              GError       **error)
{
  return dex_async_result_propagate_boolean (DEX_ASYNC_RESULT (result), error);
}

Safety Notes

One thing that Libdex handles better than GTask is ensuring that your user_data is destroyed on the proper thread. The design of Dex.Block was done in such a way that both the result and user_data are passed back to the owning thread at the same time. This ensures that your user_data will never be finalized on the wrong thread.

This article can be found in the documentation at Integrating GAsyncResult.