Happenings in GNOME

I took a different approach to directory listings in Foundry. They use GListModel as the interface but behind the scenes it is implemented with futures and and fibers.

A primary use case for a directory listing is the project tree of an IDE. Since we use GtkListView for efficient trees in GTK it we expose a GListModel. Each item in the directory is represented as a FoundryDirectoryItem which acts just a bit like a specialized GFileInfo. It contains information about all the attributes requested in the directory listing.

You can also request some other information that is not traditionally available via GFileInfo. You can request attributes that will be populated by the version control system such as if the file is modified or should be ignored.

Use a GtkFilterListModel to look at specific properties or attributes. Sort them quickly using GtkSortListModel. All of this makes implementing a project tree browser very straight forward.

Reining in the Complexity

One of the most complex things in writing a directory listing is managing updates to the directory. To manage some level of correctness here Foundry does it with a fiber in the following ordering:

• Start a file monitor on the directory and start queing up changes
• Enumerate children in the directory and add to internal list
• Start processing monitor events, starting with the backlog

Performance

There are a couple tricks to balancing the performance of new items being added and items being removed. We want both to be similarly well performing.

To do this, new items are always placed at the end of the list. We don’t care about sorting here because that will be dealt with at a higher layer (in the GtkSortListModel). That keeps adding new items quite fast because we can quickly access the end of the list. This saves us a lot of time sorting the tree just for the removal.

But if you aren’t sorting the items, how do you make removals quick? Doesn’t that become O(n)?

Well not quite. If we keep a secondary index (in this case a simple GHashTable) then we can store a key (the file’s name) which points to a stable pointer (a GSequenceIter). That lookup is O(1) and the removal from a GSequence on average are O(log n).

Big O notation is often meaningless when you’re talking about different systems. So let’s be real for a moment. Your callback from GListModel::items-changed() can have a huge impact on performance compared to the internal data structures here.

Reference Counting and Fibers

When doing this with a fiber attention must be taken to avoid over referencing the object or you risk never disposing/finalizing.

One way out of that mess is to use a GWeakRef and only request the object when it is truly necessary. That way, you can make your fiber cancel when the directory listing is disposed. In turn your monitor and other resources are automatically cleaned up.

For example:

GWeakRef *wr = g_new0 (GWeakRef, 1);
g_weak_ref_init (wr, self);
self->fiber = dex_scheduler_spawn (NULL, 0, fiber_func,
                                   wr, free_weak_ref);

And in dispose you can simply dex_clear (&self->fiber); and the fiber will automatically cancel.

In the fiber I start by creating a set of futures that will resolve when there is more work to be done. Then I release my self object followed by awaiting the future. At that point I’ll either resolve or reject from an error (including the fiber being cancelled).

If I need self again because I got an event, it’s just a g_weak_ref_get() away.

Future-based File Monitors

I’m not a huge fan of “signal based” APIs like GFileMonitor so internally FoundryFileMonitor does something a bit different. It still uses GFileMonitor internally for outstanding portability, but the exposed API uses DexFuture.

This allows you to call when you want to get the next event (which may already be queued). Call foundry_file_monotor_next() to get the next event and the future will resolve when an event has been received. This makes more complex awaiting operations feasible too (such as monitoring multiple directories for the next change).

foundry-directory-listing.c, foundry-directory-item.c, and foundry-file-monitor.c.

In addition to all the Libdex 1.0 fanfare, Foundry has also reached 1.0 for GNOME 49. That doesn’t mean it’s complete, but it does mean that what is there I feel pretty confident about from an API/ABI standpoint.

If you have a project that works in GNOME Builder, it is a good time to test it out with Foundry! To get started you need to “initialize” a Foundry project similar to what you do with Git. This creates a minimal skeleton structure that Foundry will use for the project.

cd myapp/
foundry init

At this point, you should have a .foundry/ directory in your project. Foundry will try to store everything it knows about your project in there. That includes settings, builds, cache, and tmp files. It also sets up a .gitignore file so that only the things you’re interested end up getting committed to the project’s git repository.

Now we can build the project.

foundry build

Foundry will do all the same sort of SDK setup, cross-container build pipelines, and build system management that GNOME Builder does.

To run the project, just run the following. It will handle building first if necessary too.

foundry run

If using a Flatpak manifest as your build configuration (you can check with foundry config list) then it will handle all the same sort of details that Builder does.

Building with Libfoundry

Of course the command line tool is great for when you are needing to quickly do something in a terminal. But perhaps you want to integrate Foundry features into your own application or tools? Everything in the command line tool is available via libfoundry.

To open an existing project with libfoundry, using just a directory to start, we can discover the .foundry directory location.

g_autofree char *state_dir = NULL;
DexFuture *future;

future = foundry_context_discover (g_get_current_dir (), NULL);
state_dir = dex_await_string (future, &error);

We can use that information to load the project.

g_autoptr(FoundryContext) context = NULL;
DexFuture *future;

future = foundry_context_new (state_dir, NULL, 0, NULL);
context = dex_await_object (future, &error);

To run a build we need to get the build manager service. That is available from the context with the :build-manager property.

g_autoptr(FoundryBuildManager) build_manager = NULL;

build_manager = foundry_context_dup_build_manager (context);

One thing you’ll notice in Foundry is that many getters take a reference to the returned value. This is because Foundry heavily uses fibers and it is always better to own your objects when you cross fiber suspension points. This is where the dup_ prefix comes from instead of get_.

Building the project is quite simple too. Just await until it has completed.

dex_await (foundry_build_manager_build (build_manager), &error);

Running with Libfoundry

To run is quite similar. Except you use the run manager instead of the build manager.

g_autoptr(FoundryRunManager) run_manager = NULL;

run_manager = foundry_context_dup_run_manager (context);
dex_await (foundry_run_manager_run (run_manager), &error));

There are of course more detailed ways to do this if you need precise control over things like where the project is deployed, such as to an external device. Or perhaps you want to control what command is run. The run manager and build manager also allows controlling the PTY that is used for both operations.

Hopefully that is just enough to get you excited about using the tooling. The API has been obsessed over but it could use more documentation of which this is the infancy.

I’ve talked a lot about fibers before. But what are they and how do they work?

From the programmer perspective they can feel a lot like a thread. They have a stack just like a real thread. They maintain a program counter just like a real thread. They can spill registers to the stack like a real thread.

Most importantly is that you can call into GTK from your fiber if running on the main thread. Your fiber will have been dispatched from the main loop of the GTK thread so it’s safe to do.

There are downsides too though. You have to allocate stack and guard pages for them like a real thread. They have some cost in transitioning between stacks even if fairly low these days. You also need to be mindful to own the lifecycle of pointers on your stack if you intend to “await” (e.g. suspend your fiber).

Many fibers may work together on a single thread where each runs a little bit until yielding back to the scheduler. This is called “cooperative multi-tasking” because it is up to the fibers to be cooperative and yield when appropriate.

That means that you generally should not “block” when writing code for fibers. It not only blocks your own fiber from making progress but all other fibers sharing your thread.

The way around this is to use non-blocking APIs instead of blocking calls like open() or read(). This is where combining a library for Futures and a library for Fibers makes a lot of sense. If you provide asynchronous APIs based on futures you immediately gain a natural point to yield from a fiber back to the scheduler.

The scheduler maintains two queues for fibers on a thread. That is because fibers exist in one of two (well three sort of) states.

• The first state is “runnable” meaning the fiber can make progress immediately.
• The second state is “blocked” meaning the fiber is waiting on a future to complete.
• The (sort of) third state is “finished” but in libdex it would be removed from all queues here.

When a fiber transitions from runnable to blocked its linked-list node migrates from one queue to the other. Naturally, that means we must be waiting for a completion of a future. The scheduler will register itself with the dependent future so it may be notified of completion.

Upon completion of the dependent future our fiber will move from the blocked to the runnable queue. The next GMainContext iteration will transition into the fibers stack, restore register values, set the instruction pointer and continue running.

Fibers get their stack from a pool of pre-allocated stacks. When they are discarded they return to the pool for quick re-use. If we have too many saved then we release the stacks memory back to the system. It’s all just mmap()-based stacks currently.

You might be wondering how we transition into the fibers stack from the thread. Libdex has a few different strategies for that based on the platforms it supports.

Windows, for example, has native support for fibers in their C runtime. So it uses ConvertThreadToFiber() and ConvertFiberToThread() for transitions.

On Linux and many other Unix-like systems we can use makecontext() and swapcontext() to transition. There was a time when swapcontext() was quite slow and so people used specialized assembly to do the same thing. These days I found that to be unnecessary (at least on Linux).

Another way to transition stacks is by using signalstack(), but libdex does not use that method.

Libdex fibers work on Linux, FreeBSD, macOS (both x86_64 and aarch64), Windows, and Illumos/Solaris. I believe Hurd also works but I’ve not verified that.

When your fiber first runs you will be placed at the base of your new stack. So if you found yourself in a debugger at your fibers entry point, it might look like you are one function deep. The first function from libdex would be equivalent to your _start on a regular thread.

If you await while 5 functions deep then your stacks register state will be saved and then your stack is set aside. The fiber will transition back to the scheduler where it left off. Then the original thread state is restored and the fiber scheduler can continue on to the next work item.

At the core, the fiber scheduler is really just a GSource within the GMainContext. It knows when it can flag itself is runnable. When dispatched it will wake up any number of runnable fibers.

To make sure that we don’t have to deal with extremely tricky situations fibers may not be migrated across threads. They are always pinned to the thread they were created on. If that becomes a problem it is usually better to break up your work into smaller tasks.

Another feature that has become handy is implicit fiber cancellation. A fiber is itself a future. If all code awaiting completion of your fiber have discarded interest then your fiber will be implicity cancelled.

Where this works out much better than real thread cancellation is that we already have natural exit points where we yield. So when your fiber goes to dex_await() it will get a DEX_ERROR_FIBER_CANCELLED in a GError. Usually when you get errors you propagate that rejection by returning from your fiber, easy.

If you do not want implicit fiber cancellation, you can “disown” your fiber using dex_future_disown().

In the future I’d love to approach stackless coroutines but that will be a taller order without compiler help or introducing complexity through macro madness.

VTE

• Little back-and-forth on what we can do to improve a11y further with VTE. Prototype’d a way to extract hyperlinks and provide them into AccessibleText, however that is not the right place.

  We really need an implementation of the at-spi hyperlink API in GTK so that VteTerminal may implement it.

• Merged a number of a11y fixes provided by Lukáš Tyrychtr which fix some of my original a11y implementation to follow the at-spi expectations better.

  Also added one of my own to fix a potential uint underrun when describing runs thanks to long to uint conversion.

Ptyxis

• Got a report of a bit of undesirable behavior when closing a window where close-able tabs are closed before showing the the special close dialog.

  It’s all more complex than I wish it was, because there are multiple ways we can enter that flow. Additionally, the asynchronous nature of closing tabs makes that state a bit “split brain” as it exists now. We may want to implement a “session” object to manage this stuff all in one place going forward.

• When showing multiple tabs the “visual bell” background did not cover the entire headerbar. Quick CSS fix that.

• Sebastian Wick needed a terminal with the Terminal Intents implemented so I took a crack at implementing that in Ptyxis. That allowed forward-progress on implementing the GLib side of things.

  Doing so found a couple of things we want to address in the upcoming FreeDesktop terminal intent “standard” and thankfully they’ve been moving that forward (including the Ptyxis-side).

Libpeas

• libpeas distro packaging was failing on the testsuite for Lua due to LGI not supporting girepository-2.0. Libpeas 2.x doesn’t use girepository at runtime, so this is relegated to the testsuite.

  For now, I’ve just disabled the Lua tests since they can’t possibly work due to that girepository usage.

  Some months back I provided patches for LGI to support girepository-2.0 but the project seems unmaintained and therefore hard for anyone to really say “yes merge this”.

  You can find some more information from Victoria Lacroix at https://www.vtrlx.ca/posts/2025/lgi-fork/

Builder

• They don’t get much testing during development so we had a small build regression on i686.

• Spent a bunch of time on what will hopefully become Builder 50. This is a major rewrite on top of Foundry, Foundry-Gtk, and what will become Foundry-Adw in the 1.1 release of Foundry.

  It is hard to describe how remarkably small this will make Builder as a project compared to previous versions. So much complexity in the Builder code-base was in response to all the async/finish flows required to keep things smooth. But now that we have libdex and libfoundry, it’s just so easy to implement that sort of magic.

Libdex

• Sebastian Wick had some great ideas on integrating libdex with gdbus-codegen. Discussed a few different ways we could go forward to make something ergonomic. It would be nice to implement both consuming as proxies and providing as skeletons with futures.

Libpanel

• I noticed that the bottom corner buttons in Builder were not aligning with the external rounded corners of the window. That was fallout from a previous cleanup so fixed that up before release.

Template-GLib

• Thanks to early testing by distributors we found an issue in the testsuite which was exercising GObject Introspection integration. It broke on 32-bit because the gsize parameters are different and I never implemented auto-casting of integers of different sizes.

  So if you did a call like GLib.utf8_substring(str, i64(3), i64(-1)) and were on 32-bit that would fail since the offset/length parameters are glong which is sized differently.

  The quick fix that will still allow for errors to propagate was to implement auto up/down casting of numbers so long as they will fit in the destination type. Otherwise, you’ll fail the cast operation and errors propagate as necessary.

  This fix landed shortly before 3.38.0 but was tested on numerous architectures before release.

Foundry

• Learned that even command-line tools get appdata now days so I went ahead and implemented that for the foundry CLI tool.

• We now dynamically link foundry binary. Originally I had plans to statically link it so that we can send it to a remote system and run it as a bridge. Since we aren’t there yet anyway, it doesn’t make sense to enforce that and/or make distributions patch it out.

• FoundryBuildManager got a :busy property which makes it much easier to implement UI like Builder has where you show a build button or a stop button based on status.

  It also got a new stop() method/GAction for that too. This is a lot harder than it looks because you need to plumb through a cancellable to all of the build pipeline API which is going to be awaiting the first future of [cancellable, some_op].

  Was extremely happy to see it work on the first try which means I’ve done something right in the libdex design.

• Once I did the above, adding a rebuild action and method was quite easy. We have all the necessary plumbing to either call an action or await a method future and get the same result.

• A last minute API was added to create a producer from a consumer PTY fd. Hopefully using this new nomenclature is not so confusing for people used to ancient PTY terminology which I know is extremely confusing to begin with. But we gotta move past those antiquated terms (which I wont repeat here) as they are both morally wrong and technically inaccurate.

• The FoundryFileManager can now automatically discover content-type when provided a filename. This vastly simplifies API usage when you have one-or-the-other to get a symbolic icon. Since we override many icons from the system, that is just a necessary abstraction.

• FoundrySearchResult got an :icon property which means it’s basically usable to search UI now. Though there are not many FoundrySearchProviders yet as they will land for 1.1.

• A new context.file-manager-show GAction is provided which allows you to pass a uri as a "s"-typed GVariant. I feel stupid for not doing this a decade ago in Builder and quite frankly, it should probably just exist in GTK.

• Libpanel/Builder has this nice “Action Muxer” API for years now and that is exported in Foundry too so we can use it in libfoundry-gtk and libfoundry-adw. It’s extremely handy when the built-in action support in GTK is not enough.

• Foundry has a SQLite database of extended attributes on URIs when the underlying file-system does not support extended attributes. I made a little boo-boo there so made that to actually work.

• Talked to dmalcom about supporting SARIF in Foundry for GNOME 50. They (the GCC project) would really like to see more consumers of it and Foundry is an obvious place to put that.

  It doesn’t look terribly difficult and it should allow us to drop the whole “regex parsing of PTY content” if done right.

Foundry-Gtk

• Fixed palette parsing and color application to VteTerminal.

• Imported all the Ptyxis terminal palettes which can now be represented as a FoundryTerminalPaletteSet and FoundryTerminalPalette.

  Ultimately, what this means is if you link against libfoundry and libfoundry-gtk you could functionally create your own Ptyxis replacement in very little code (assuming you don’t use an agent on the host OS like Ptyxis does).

  Just use the FoundrySdk as your container abstraction (which you can query using FoundrySdkManager) and FoundryTerminal with FoundryTerminalPaletteSet.

• You can now list/find available palettes with a GListModel API in the form of foundry_terminal_list_palette_sets() and foundry_terminal_find_palette_set(). Each set has an easy light/dark property you can use based on your needs.

• The file-search plugin now properly implements the load() vfunc for search results so you can implement preview/opening in apps.

Foundry-Adw

• A ton of work on the workspace, page, and panel APIs for 1.1. I’m really trying to find a way to re-use this across a number of applications such as Builder, Drafting, Sysprof, and more.

Releases

• gnome-text-editor 49.0

• gtksourceview 5.18.0

• sysprof 49.0

• gnome-builder 49.0

• ptyxis 49.0

• libdex 1.0.0

• foundry 1.0.0

• libpanel 1.10.2

• template-glib 3.38.0

• d-spy 49.0 (thanks to Jordan for the 49.1 w/ CI fixed)

• gom 0.5.4

• libpeas 2.2.0

• manuals 49.0

Other

• Team coffee hour, shared some battle wounds of trying to use AI for a pahole wrapper which fixed all my class structs to be cacheline aligned. Was just faster to write the damn code.

There are often needs to integrate with blocking APIs that do not fit well into the async or future-based models of the GNOME ecosystem. In those cases, you may want to use a dedicated thread for blocking calls so that you do not disrupt main loops, timeouts, or fiber scheduling.

This is ideal when doing things like interacting with libflatpak or even libgit2.

Creating a Dedicated Thread

Use the dex_thread_spawn() function to spawn a new thread. When the thread completes the resulting future will either resolve or reject.

typedef DexFuture *(*DexThreadFunc) (gpointer user_data);

DexFuture *future = dex_thread_spawn ("[my-thredad]", thread_func, thread_data,
                                      (GDestroyNotify)thread_data_free);

Waiting for Future Completion

Since dedicated threads do not have a Dex.Scheduler on them and are not a fiber, you may not await futures. Awaiting would suspend a fiber stack but there is no such fiber to suspend.

To make integration easier, you may use dex_thread_wait_for() to wait for a future to complete. The mechanism used in this case is a mutex and condition variable which will be signaled when the dependent future completes.

Previously, previously, and previously.

The Gio.IOStream APIs already provide robust support for asynchronous IO. The common API allows for different types of implementation based on the stream implementation.

Libdex provides wrappers for various APIs. Coverage is not complete but we do expect additional APIs to be covered in future releases.

File Management

See dex_file_copy() for copying files.

See dex_file_delete() for deleting files.

See dex_file_move() for moving files.

File Attributes

See dex_file_query_info() and dex_file_query_file_type(), and dex_file_query_exists() for basic querying.

You can set file attributes using dex_file_set_attributes().

Directories

You can create a directory or hierarchy of directories using dex_file_make_directory() and dex_file_make_directory_with_parents() respectively.

Enumerating Files

You can create a file enumerator for a directory using dex_file_enumerate_children().

You can also asynchronously enumerate the files of that directory using dex_file_enumerator_next_files() which will resolve to a g_autolist(GFileInfo) of infos.

Reading and Writing Files

The dex_file_read() will provide a Gio.FileInputStream which can be read from.

A simpler interface to get the bytes of a file is provided via dex_file_load_contents_bytes().

The dex_file_replace() will replace a file on disk providing a Gio.FileOutputStream to write to. The dex_file_replace_contents_bytes() provides a simplified API for this when the content is readily available.

Reading Streams

See dex_input_stream_read(), dex_input_stream_read_bytes(), dex_input_stream_skip(), and dex_input_stream_close() for working with input streams asynchronously.

Writing Streams

See dex_output_stream_write(), dex_output_stream_write_bytes(), dex_output_stream_splice(), and dex_output_stream_close() for writing to streams asynchronously.

Sockets

The dex_socket_listener_accept(), dex_socket_client_connect(), and dex_resolver_lookup_by_name() may be helpful when writing socket servers and clients.

D-Bus

Light integration exists for D-Bus to perform asychronous method calls.

See dex_dbus_connection_call(), dex_dbus_connection_call_with_unix_fd_list(), dex_dbus_connection_send_message_with_reply() and dex_dbus_connection_close().

We expect additional support for D-Bus to come at a later time.

Subprocesses

You can await completion of a subprocess using dex_subprocess_wait_check().

Foundry uses some helpers to do UTF-8 communication but I’d like to bring that to libdex in an improved way post-1.0.

Asynchronous IO with File Descriptors

Gio.IOStream and related APIs provides much opertunity for streams to be used asynchronously. There may be cases where you want similar behavior with traditional file-descriptors.

Libdex provides a set of AIO-like functions for traditional file-descriptors which may be backed with more efficient mechanisms.

Gio.IOStream typically uses a thread pool of blocking IO operations on Linux and other operating systems because that was the fastest method when the APIs were created. However, on some operating systems such as Linux, faster methods finally exist.

On Linux, io_uring can be used for asynchronous IO and is provided in the form of a Dex.Future.

Asynchronous Reads

DexFuture *dex_aio_read  (DexAioContext *aio_context,
                          int            fd,
                          gpointer       buffer,
                          gsize          count,
                          goffset        offset);

Use the dex_aio_read() function to read from a file-descriptor. The result will be a future that resolves to a gint64 containing the number of bytes read.

If there was a failure, the future will reject using the appropriate error code.

Your buffer must stay alive for the duration of the asynchronous read. One easy way to make that happen is to wrap the resulting future in a dex_future_then() which stores the buffer as user_data and releases it when finished.

If you are doing buffer pooling, more effort may be required.

Asynchronous Writes

DexFuture *dex_aio_write (DexAioContext *aio_context,
                          int            fd,
                          gconstpointer  buffer,
                          gsize          count,
                          goffset        offset);

A similar API exists as dex_aio_read() but for writing. It too will resolve to a gint64 containing the number of bytes written.

buffer must be kept alive for the duration of the call and it is the callers responsibility to do so.

You can find this article in the documentation under Asynchronous IO.

Previously, and previously.

Schedulers

The Dex.Scheduler is responsible for running work items on a thread. This is performed by integrating with the threads GMainContext. The main thread of your application will have a Dex.MainScheduler as the assigned scheduler.

The scheduler manages callbacks such as work items created with Dex.Scheduler.push(). This can include blocks, fibers, and application provided work.

You can get the default scheduler for the application’s main thread using Dex.Scheduler.get_default(). The current thread’s scheduler can be retrieved with Dex.Scheduler.ref_thread_default().

Thread Pool Scheduling

Libdex manages a thread pool which may be retrieved using Dex.ThreadPoolScheduler.get_default().

The thread pool scheduler will manage a number of threads that is deemed useful based on the number of CPU available. When io_uring is used, it will also restrict the number of workers to the number of uring available.

Work items created from outside of the thread pool are placed into a global queue. Thread pool workers will take items from the global queue when they have no more items to process.

To avoid “thundering herd” situations often caused by global queues and thread pools a pollable semaphore is used. On Linux, specifically, io_uring and eventfd combined with EFD_SEMAPHORE allow waking up a single worker when a work item is queued.

All thread pool workers have a local Dex.Scheduler so use of timeouts and other GSource features continue to work.

If you need to interact with long-blocking API calls it is better to use Dex.thread_spawn() rather than a thread pool thread.

Thread pool workers use a work-stealing wait-free queue which allows the worker to push work items onto one side of the queue quickly. Doing so also helps improve cacheline effectiveness.

Fibers

Fibers are a type of stackfull co-routine. A new stack is created and a trampoline is performed onto the stack from the current thread.

Use Dex.Scheduler.spawn() to create a new fiber.

When a fiber calls one of the Dex.Future.await() functions or when it returns the fiber is suspended and execution returns to the scheduler.

By default, fibers have a 128-kb stack with a guard page at the end. Fiber stacks are pooled so that they may be reused during heavy use.

Fibers are a Dex.Future which means you can await the completion of a fiber just like any other future.

Note that fibers are pinned to a scheduler. They will not be migrated between schedulers even when a thread pool is in use.

Fiber Cancellation

Fibers may be cancelled if the fiber has been discarded by all futures awaiting completion. Fibers will always exit through a natural exit point such as a pending “await”. All attempts to await will reject with error once a fiber has been cancelled.

If you want to ignore cancellation of fibers, use Dex.Future.disown() on the fiber after creation.

This article can be found at scheduling in the libdex documentation.

Previously.

Historically if you wanted to do asynchronous work in GObject-based applications you would use GAsyncReadyCallback and GAsyncResult.

There are two ways to integrate with this form of asynchronous API.

In one direction, you can consume this historical API and provide the result as a DexFuture. In the other direction, you can provide this API in your application or library but implement it behind the scenes with DexFuture.

Converting GAsyncResult to Futures

A typical case to integrate, at least initially, is to extract the result of a GAsyncResult and propagate it to a future.

One way to do that is with a Dex.Promise which will resolve or reject from your async callback.

static void
my_callback (GObject      *object,
             GAsyncResult *result,
             gpointer      user_data)
{
  g_autoptr(DexPromise) promise = user_data;
  g_autoptr(GError) error = NULL;

  if (thing_finish (THING (object), result, &error))
    dex_promise_resolve_boolean (promise, TRUE);
  else
    dex_promise_reject (promise, g_steal_pointer (&error));
}

DexFuture *
my_wrapper (Thing *thing)
{
  DexPromise *promise = dex_promise_new_cancellable ();

  thing_async (thing,
               dex_promise_get_cancellable (promise),
               my_callback,
               dex_ref (promise));

  return DEX_FUTURE (promise);
}

Implementing AsyncResult with Futures

In some cases you may not want to “leak” into your API that you are using DexFuture. For example, you may want to only expose a traditional GIO API or maybe even clean up legacy code.

For these cases use Dex.AsyncResult. It is designed to feel familiar to those that have used GTask.

Dex.AsyncResult.new() allows taking the typical cancellable, callback, and user_data parameters similar to GTask.

Then call Dex.AsyncResult.await() providing the future that will resolve or reject with error. One completed, the users provided callback will be executed within the active scheduler at time of creation.

From your finish function, call the appropriate propgate API such as Dex.AsyncResult.propagate_int().

void
thing_async (Thing               *thing,
             GCancellable        *cancellable,
             GAsyncReadyCallback  callback,
             gpointer             user_data)
{
  g_autoptr(DexAsyncResult) result = NULL;

  result = dex_async_result_new (thing, cancellable, callback, user_data);
  dex_async_result_await (result, dex_future_new_true ());
}

gboolean
thing_finish (Thing         *thing,
              GAsyncResult  *result,
              GError       **error)
{
  return dex_async_result_propagate_boolean (DEX_ASYNC_RESULT (result), error);
}

Safety Notes

One thing that Libdex handles better than GTask is ensuring that your user_data is destroyed on the proper thread. The design of Dex.Block was done in such a way that both the result and user_data are passed back to the owning thread at the same time. This ensures that your user_data will never be finalized on the wrong thread.

This article can be found in the documentation at Integrating GAsyncResult.

A couple years ago I spent a great deal of time in the waiting room of an allergy clinic. So much that I finally found the time to write a library that was meant to be a followup to libgtask/libiris libraries I wrote nearly two decades ago. A lot has changed in Linux since then and I felt that maybe this time, I could get it “right”.

This will be a multi-part series, but today lets focus on terminology so we have a common language to communicate.

Futures

A future is a container that will eventually contain a result or an error.

Programmers often use the words “future” and “promise” interchangeably. Libdex tries, when possible, to follow the academic nomenclature for futures. That is to say that a future is the interface and promise is a type of future.

Futures exist in one of three states. The first state is pending. A future exists in this state until it has either rejected or resolved.

The second state is resolved. A future reaches this state when it has successfully obtained a value.

The last third state is rejected. If there was a failure to obtain a value a future will be in this state and contain a GError representing such failure.

Promises and More

A promise is a type of future that allows the creator to set the resolved value or error. This is a common type of future to use when you are integrating with things that are not yet integrated with Libdex.

Other types of futures also exist.

/* resolve to "true" */
DexPromise *good = dex_promise_new ();
dex_promise_resolve_boolean (good, TRUE);

/* reject with error */
DexPromise *bad = dex_promise_new ();
dex_promise_reject (good,
                    g_error_new (G_IO_ERROR,
                                 G_IO_ERROR_FAILED,
                                 "Failed"));

Static Futures

Sometimes you already know the result of a future upfront.
The DexStaticFuture is used in this case.
Various constructors for DexFuture will help you create one.

For example, Dex.Future.new_take_object() will create a static future for a GObject-derived instance.

DexFuture *future = dex_future_new_for_int (123);

Blocks

One of the most commonly used types of futures in Libdex is a DexBlock.

A DexBlock is a callback that is run to process the result of a future. The block itself is also a future meaning that you can chain these blocks together into robust processing groups.

“Then” Blocks

The first type of block is a “then” block which is created using Dex.Future.then(). These blocks will only be run if the dependent future resolves with a value. Otherwise, the rejection of the dependent future is propagated to the block.

static DexFuture *
further_processing (DexFuture *future,
                    gpointer   user_data)
{
  const GValue *result = dex_promise_get_value (future, NULL);

  /* since future is completed at this point, you can also use
   * the simplified "await" API. Otherwise you'd get a rejection
   * for not being on a fiber. (More on that later).
   */
  g_autoptr(GObject) object = dex_await_object (dex_ref (future), NULL);

  return dex_ref (future);
}

“Catch” Blocks

Since some futures may fail, there is value in being able to “catch” the failure and resolve it.

Use Dex.Future.catch() to handle the result of a rejected future and resolve or reject it further.

static DexFuture *
catch_rejection (DexFuture *future,
                 gpointer   user_data)
{
  g_autoptr(GError) error = NULL;

  dex_future_get_value (future, &error);

  if (g_error_matches (error, G_IO_ERROR, G_IO_ERROR_NOT_FOUND))
    return dex_future_new_true ();

  return dex_ref (future);
}

“Finally” Blocks

There may be times when you want to handle completion of a future whether it resolved or rejected. For this case, use a “finally” block by calling Dex.Future.finally().

Infinite Loops

If you find you have a case where you want a DexBlock to loop indefinitely, you can use the _loop variants of the block APIs.

See Dex.Future.then_loop(), Dex.Future.catch_loop(), or Dex.Future.finally_loop(). This is generally useful when your block’s callback will begin the next stage of work as the result of the callback.

Future Sets

A FutureSet is a type of future that is the composition of multiple futures. This is an extremely useful construct because it allows you to do work concurrently and then process the results in a sort of “reduce” phase.

For example, you could make a request to a database, cache server, and a timeout and process the first that completes.

There are multiple types of future sets based on the type of problem you want to solve.

Dex.Future.all() can be used to resolve when all dependent futures have resolved, otherwise it will reject with error once they are complete.
If you want to reject as soon as the first item rejects, Dex.Future.all_race() will get you that behavior.

Other useful Dex.FutureSet construtors include Dex.Future.any() and Dex.Future.first.

/* Either timeout or propagate result of cache/db query */
return dex_future_first (dex_timeout_new_seconds (60),
                         dex_future_any (query_db_server (),
                                         query_cache_server (),
                                         NULL),
                         NULL);

Cancellable

Many programmers who use GTK and GIO are familiar with GCancellable. Libdex has something similar in the form of DexCancellable. However, in the Libdex case, DexCancellable is a future.

It allows for convenient grouping with other futures to perform cancellation when the Dex.Cancellable.cancel() method is called.

It can also integrate with GCancellable when created using Dex.Cancellable.new_from_cancellable().

A DexCancellable will only ever reject.

DexFuture *future = dex_cancellable_new ();
dex_cancellable_cancel (DEX_CANCELLABLE (future));

Timeouts

A timeout may be represented as a future.
In this case, the timeout will reject after a time period has passed.

A DexTimeout will only ever reject.

This future is implemented ontop of GMainContext via API like g_timeout_add().

DexFuture *future = dex_timeout_new_seconds (60);

Unix Signals

Libdex can represent unix signals as a future. That is to say that the future will resolve to an integer of the signal number when that signal has been raised.

This is implemented using g_unix_signal_source_new() and comes with all the same restrictions.

Delayed

Sometimes you may run into a case where you want to gate the result of a future until a specific moment.

For this case, DexDelayed allows you to wrap another future and decide when to “uncork” the result.

DexFuture *delayed = dex_delayed_new (dex_future_new_true ());
dex_delayed_release (DEX_DELAYED (delayed));

Fibers

Another type of future is a “fiber”.

More care will be spent on fibers later on but suffice to say that the result of a fiber is easily consumable as a future via DexFiber.

DexFuture *future = dex_scheduler_spawn (NULL, 0, my_fiber, state, state_free);

Cancellation Propagation

Futures within your application will enevitably depend on other futures.

If all of the futures depending on a future have been released, the dependent future will have the opportunity to cancel itself. This allows for cascading cancellation so that unnecessary work may be elided.

You can use Dex.Future.disown() to ensure that a future will continue to be run even if the dependent futures are released.

Schedulers

Libdex requires much processing that needs to be done on the main loop of a thread. This is generally handled by a DexScheduler.

The main thread of an application has the default sheduler which is a DexMainScheduler.

Libdex also has a managed thread pool of schedulers via the DexThreadPoolScheduler.

Schedulers manage short tasks, executing DexBlock when they are ready, finalizing objects on their owning thread, and running fibers.

Schedulers integrate with the current threads GMainContext via GSource making it easy to use Libdex with GTK and Clutter-based applications.

Channels

DexChannel is a higher-level construct built on futures that allow passing work between producers and consumers. They are akin to Go channels in that they have a read and a write side. However, they are much more focused on integrating well with DexFuture.

You can find this article in the Libdex documentation under terminology.

Foundry

• Spent a bit of time working out how we can enable selection of app patterns in Foundry. The goal here would be to have some very common libadwaita usage patterns available for selection in the new-project guide.

  Ultimately it will rely on FoundryInputCombo/FoundryInputChoice but we’ll have to beef it up to support iconography.

• Finish up a couple speed-runs so they can be uploaded to to gitlab. chergert/assist and chergert/staged are there and can serve as an example of how to use the libfoundry API.

• A big portion of this week has been figuring out how I want to expose tweaks from libfoundry into applications. There are a lot of caveats here which make it somewhat tricky.

  For example, not every application will need every toggle, so we need a way to filter them. GListModel is probably our easiest way out with this, but we’ll likely need a bit more control over provenance of tweaks here for filtering.

• Tweak engine has a new “path” design which allows us to dynamically query available tweaks as you dive down deeper into the UI. This is primarily to help avoid some of the slower parts of the tweaks engine in GNOME Builder.

  Also, to help support this, we can define tweaks using static data which allows for registration/query much faster. For example, there is no need to do UI merging since that can happen automatically.

  There are also cases where you may need to register tweaks which are more complex than simple GSettings. We should be able to accommodate that just fine.

• Added API for add/remove to Git stage. Much simpler than using the libgit2 index APIs directly, for the common things.

  We may want to add a “Stage” API at some point. But for now, the helpers do the job for the non-partial-stage use-case.

  Just foundry_git_vcs_stage_entry (vcs, entry, contents) where the entry comes from your status list. You can retrieve that with foundry_git_vcs_list_status (vcs). If you are brave enough to do your own diff/patch you can implement staging lines with this too.

  However, that is probably where we want an improved stage helper.

• Added API for creating commits.

  Much easier to do foundry_git_vcs_commit (vcs, msg, name, email) than the alternatives.

• Running LLM tools can now be done through the conversation object which allows for better retention of state. Specifically because that can allow the conversation to track a history object other than a simple wrapped FoundryLlmMessage.

  For example, some message subclasses may have extra state like a “directory listing” which UI could use to show something more interesting than some text.

• Simplify creating UI around FoundryLlmConversation with a new busy property that you can use to cancel your in-flight futures.

• Fixed an issue where running the LLM tool (via subprocess) would not proxy the request back to the parent UI process. Fixing that means that you can update the application UI when the conversation is running a tool. If you were, for example, to cancel the build then the tool would get canceled too.

• Made the spellcheck integration work the same as we do in Builder and Text Editor which is to move the cursor first on right-click before showing corrections.

Libspelling

• New release with some small bugfixes.

Levers

• Another speed run application which is basically the preferences part of Builder but on top of Foundry. The idea here is that you can just drop into a project and tweak most aspects of it.

  Not intended to be a “real” application like the other speed-runs, but at least it helps ensure that the API is relatively useful.