Status Week 37

VTE

  • Little back-and-forth on what we can do to improve a11y further with VTE. Prototype’d a way to extract hyperlinks and provide them into AccessibleText, however that is not the right place.

    We really need an implementation of the at-spi hyperlink API in GTK so that VteTerminal may implement it.

  • Merged a number of a11y fixes provided by Lukáš Tyrychtr which fix some of my original a11y implementation to follow the at-spi expectations better.

    Also added one of my own to fix a potential uint underrun when describing runs thanks to long to uint conversion.

Ptyxis

  • Got a report of a bit of undesirable behavior when closing a window where close-able tabs are closed before showing the the special close dialog.

    It’s all more complex than I wish it was, because there are multiple ways we can enter that flow. Additionally, the asynchronous nature of closing tabs makes that state a bit “split brain” as it exists now. We may want to implement a “session” object to manage this stuff all in one place going forward.

  • When showing multiple tabs the “visual bell” background did not cover the entire headerbar. Quick CSS fix that.

  • Sebastian Wick needed a terminal with the Terminal Intents implemented so I took a crack at implementing that in Ptyxis. That allowed forward-progress on implementing the GLib side of things.

    Doing so found a couple of things we want to address in the upcoming FreeDesktop terminal intent “standard” and thankfully they’ve been moving that forward (including the Ptyxis-side).

Libpeas

  • libpeas distro packaging was failing on the testsuite for Lua due to LGI not supporting girepository-2.0. Libpeas 2.x doesn’t use girepository at runtime, so this is relegated to the testsuite.

    For now, I’ve just disabled the Lua tests since they can’t possibly work due to that girepository usage.

    Some months back I provided patches for LGI to support girepository-2.0 but the project seems unmaintained and therefore hard for anyone to really say “yes merge this”.

    You can find some more information from Victoria Lacroix at https://www.vtrlx.ca/posts/2025/lgi-fork/

Builder

  • They don’t get much testing during development so we had a small build regression on i686.

  • Spent a bunch of time on what will hopefully become Builder 50. This is a major rewrite on top of Foundry, Foundry-Gtk, and what will become Foundry-Adw in the 1.1 release of Foundry.

    It is hard to describe how remarkably small this will make Builder as a project compared to previous versions. So much complexity in the Builder code-base was in response to all the async/finish flows required to keep things smooth. But now that we have libdex and libfoundry, it’s just so easy to implement that sort of magic.

Libdex

  • Sebastian Wick had some great ideas on integrating libdex with gdbus-codegen. Discussed a few different ways we could go forward to make something ergonomic. It would be nice to implement both consuming as proxies and providing as skeletons with futures.

Libpanel

  • I noticed that the bottom corner buttons in Builder were not aligning with the external rounded corners of the window. That was fallout from a previous cleanup so fixed that up before release.

Template-GLib

  • Thanks to early testing by distributors we found an issue in the testsuite which was exercising GObject Introspection integration. It broke on 32-bit because the gsize parameters are different and I never implemented auto-casting of integers of different sizes.

    So if you did a call like GLib.utf8_substring(str, i64(3), i64(-1)) and were on 32-bit that would fail since the offset/length parameters are glong which is sized differently.

    The quick fix that will still allow for errors to propagate was to implement auto up/down casting of numbers so long as they will fit in the destination type. Otherwise, you’ll fail the cast operation and errors propagate as necessary.

    This fix landed shortly before 3.38.0 but was tested on numerous architectures before release.

Foundry

  • Learned that even command-line tools get appdata now days so I went ahead and implemented that for the foundry CLI tool.

  • We now dynamically link foundry binary. Originally I had plans to statically link it so that we can send it to a remote system and run it as a bridge. Since we aren’t there yet anyway, it doesn’t make sense to enforce that and/or make distributions patch it out.

  • FoundryBuildManager got a :busy property which makes it much easier to implement UI like Builder has where you show a build button or a stop button based on status.

    It also got a new stop() method/GAction for that too. This is a lot harder than it looks because you need to plumb through a cancellable to all of the build pipeline API which is going to be awaiting the first future of [cancellable, some_op].

    Was extremely happy to see it work on the first try which means I’ve done something right in the libdex design.

  • Once I did the above, adding a rebuild action and method was quite easy. We have all the necessary plumbing to either call an action or await a method future and get the same result.

  • A last minute API was added to create a producer from a consumer PTY fd. Hopefully using this new nomenclature is not so confusing for people used to ancient PTY terminology which I know is extremely confusing to begin with. But we gotta move past those antiquated terms (which I wont repeat here) as they are both morally wrong and technically inaccurate.

  • The FoundryFileManager can now automatically discover content-type when provided a filename. This vastly simplifies API usage when you have one-or-the-other to get a symbolic icon. Since we override many icons from the system, that is just a necessary abstraction.

  • FoundrySearchResult got an :icon property which means it’s basically usable to search UI now. Though there are not many FoundrySearchProviders yet as they will land for 1.1.

  • A new context.file-manager-show GAction is provided which allows you to pass a uri as a "s"-typed GVariant. I feel stupid for not doing this a decade ago in Builder and quite frankly, it should probably just exist in GTK.

  • Libpanel/Builder has this nice “Action Muxer” API for years now and that is exported in Foundry too so we can use it in libfoundry-gtk and libfoundry-adw. It’s extremely handy when the built-in action support in GTK is not enough.

  • Foundry has a SQLite database of extended attributes on URIs when the underlying file-system does not support extended attributes. I made a little boo-boo there so made that to actually work.

  • Talked to dmalcom about supporting SARIF in Foundry for GNOME 50. They (the GCC project) would really like to see more consumers of it and Foundry is an obvious place to put that.

    It doesn’t look terribly difficult and it should allow us to drop the whole “regex parsing of PTY content” if done right.

Foundry-Gtk

  • Fixed palette parsing and color application to VteTerminal.

  • Imported all the Ptyxis terminal palettes which can now be represented as a FoundryTerminalPaletteSet and FoundryTerminalPalette.

    Ultimately, what this means is if you link against libfoundry and libfoundry-gtk you could functionally create your own Ptyxis replacement in very little code (assuming you don’t use an agent on the host OS like Ptyxis does).

    Just use the FoundrySdk as your container abstraction (which you can query using FoundrySdkManager) and FoundryTerminal with FoundryTerminalPaletteSet.

  • You can now list/find available palettes with a GListModel API in the form of foundry_terminal_list_palette_sets() and foundry_terminal_find_palette_set(). Each set has an easy light/dark property you can use based on your needs.

  • The file-search plugin now properly implements the load() vfunc for search results so you can implement preview/opening in apps.

Foundry-Adw

  • A ton of work on the workspace, page, and panel APIs for 1.1. I’m really trying to find a way to re-use this across a number of applications such as Builder, Drafting, Sysprof, and more.

Releases

  • gnome-text-editor 49.0

  • gtksourceview 5.18.0

  • sysprof 49.0

  • gnome-builder 49.0

  • ptyxis 49.0

  • libdex 1.0.0

  • foundry 1.0.0

  • libpanel 1.10.2

  • template-glib 3.38.0

  • d-spy 49.0 (thanks to Jordan for the 49.1 w/ CI fixed)

  • gom 0.5.4

  • libpeas 2.2.0

  • manuals 49.0

Other

  • Team coffee hour, shared some battle wounds of trying to use AI for a pahole wrapper which fixed all my class structs to be cacheline aligned. Was just faster to write the damn code.

Dedicated Threads with Futures

There are often needs to integrate with blocking APIs that do not fit well into the async or future-based models of the GNOME ecosystem. In those cases, you may want to use a dedicated thread for blocking calls so that you do not disrupt main loops, timeouts, or fiber scheduling.

This is ideal when doing things like interacting with libflatpak or even libgit2.

Creating a Dedicated Thread

Use the dex_thread_spawn() function to spawn a new thread. When the thread completes the resulting future will either resolve or reject.

typedef DexFuture *(*DexThreadFunc) (gpointer user_data);

DexFuture *future = dex_thread_spawn ("[my-thredad]", thread_func, thread_data,
                                      (GDestroyNotify)thread_data_free);

Waiting for Future Completion

Since dedicated threads do not have a Dex.Scheduler on them and are not a fiber, you may not await futures. Awaiting would suspend a fiber stack but there is no such fiber to suspend.

To make integration easier, you may use dex_thread_wait_for() to wait for a future to complete. The mechanism used in this case is a mutex and condition variable which will be signaled when the dependent future completes.

Asynchronous IO with Libdex

Previously, previously, and previously.

The Gio.IOStream APIs already provide robust support for asynchronous IO. The common API allows for different types of implementation based on the stream implementation.

Libdex provides wrappers for various APIs. Coverage is not complete but we do expect additional APIs to be covered in future releases.

File Management

See dex_file_copy() for copying files.

See dex_file_delete() for deleting files.

See dex_file_move() for moving files.

File Attributes

See dex_file_query_info() and dex_file_query_file_type(), and dex_file_query_exists() for basic querying.

You can set file attributes using dex_file_set_attributes().

Directories

You can create a directory or hierarchy of directories using dex_file_make_directory() and dex_file_make_directory_with_parents() respectively.

Enumerating Files

You can create a file enumerator for a directory using dex_file_enumerate_children().

You can also asynchronously enumerate the files of that directory using dex_file_enumerator_next_files() which will resolve to a g_autolist(GFileInfo) of infos.

Reading and Writing Files

The dex_file_read() will provide a Gio.FileInputStream which can be read from.

A simpler interface to get the bytes of a file is provided via dex_file_load_contents_bytes().

The dex_file_replace() will replace a file on disk providing a Gio.FileOutputStream to write to. The dex_file_replace_contents_bytes() provides a simplified API for this when the content is readily available.

Reading Streams

See dex_input_stream_read(), dex_input_stream_read_bytes(), dex_input_stream_skip(), and dex_input_stream_close() for working with input streams asynchronously.

Writing Streams

See dex_output_stream_write(), dex_output_stream_write_bytes(), dex_output_stream_splice(), and dex_output_stream_close() for writing to streams asynchronously.

Sockets

The dex_socket_listener_accept(), dex_socket_client_connect(), and dex_resolver_lookup_by_name() may be helpful when writing socket servers and clients.

D-Bus

Light integration exists for D-Bus to perform asychronous method calls.

See dex_dbus_connection_call(), dex_dbus_connection_call_with_unix_fd_list(), dex_dbus_connection_send_message_with_reply() and dex_dbus_connection_close().

We expect additional support for D-Bus to come at a later time.

Subprocesses

You can await completion of a subprocess using dex_subprocess_wait_check().

Foundry uses some helpers to do UTF-8 communication but I’d like to bring that to libdex in an improved way post-1.0.

Asynchronous IO with File Descriptors

Gio.IOStream and related APIs provides much opertunity for streams to be used asynchronously. There may be cases where you want similar behavior with traditional file-descriptors.

Libdex provides a set of AIO-like functions for traditional file-descriptors which may be backed with more efficient mechanisms.

Gio.IOStream typically uses a thread pool of blocking IO operations on Linux and other operating systems because that was the fastest method when the APIs were created. However, on some operating systems such as Linux, faster methods finally exist.

On Linux, io_uring can be used for asynchronous IO and is provided in the form of a Dex.Future.

Asynchronous Reads

DexFuture *dex_aio_read  (DexAioContext *aio_context,
                          int            fd,
                          gpointer       buffer,
                          gsize          count,
                          goffset        offset);

Use the dex_aio_read() function to read from a file-descriptor. The result will be a future that resolves to a gint64 containing the number of bytes read.

If there was a failure, the future will reject using the appropriate error code.

Your buffer must stay alive for the duration of the asynchronous read. One easy way to make that happen is to wrap the resulting future in a dex_future_then() which stores the buffer as user_data and releases it when finished.

If you are doing buffer pooling, more effort may be required.

Asynchronous Writes

DexFuture *dex_aio_write (DexAioContext *aio_context,
                          int            fd,
                          gconstpointer  buffer,
                          gsize          count,
                          goffset        offset);

A similar API exists as dex_aio_read() but for writing. It too will resolve to a gint64 containing the number of bytes written.

buffer must be kept alive for the duration of the call and it is the callers responsibility to do so.

You can find this article in the documentation under Asynchronous IO.

Scheduling & Fibers

Previously, and previously.

Schedulers

The Dex.Scheduler is responsible for running work items on a thread. This is performed by integrating with the threads GMainContext. The main thread of your application will have a Dex.MainScheduler as the assigned scheduler.

The scheduler manages callbacks such as work items created with Dex.Scheduler.push(). This can include blocks, fibers, and application provided work.

You can get the default scheduler for the application’s main thread using Dex.Scheduler.get_default(). The current thread’s scheduler can be retrieved with Dex.Scheduler.ref_thread_default().

Thread Pool Scheduling

Libdex manages a thread pool which may be retrieved using Dex.ThreadPoolScheduler.get_default().

The thread pool scheduler will manage a number of threads that is deemed useful based on the number of CPU available. When io_uring is used, it will also restrict the number of workers to the number of uring available.

Work items created from outside of the thread pool are placed into a global queue. Thread pool workers will take items from the global queue when they have no more items to process.

To avoid “thundering herd” situations often caused by global queues and thread pools a pollable semaphore is used. On Linux, specifically, io_uring and eventfd combined with EFD_SEMAPHORE allow waking up a single worker when a work item is queued.

All thread pool workers have a local Dex.Scheduler so use of timeouts and other GSource features continue to work.

If you need to interact with long-blocking API calls it is better to use Dex.thread_spawn() rather than a thread pool thread.

Thread pool workers use a work-stealing wait-free queue which allows the worker to push work items onto one side of the queue quickly. Doing so also helps improve cacheline effectiveness.

Fibers

Fibers are a type of stackfull co-routine. A new stack is created and a trampoline is performed onto the stack from the current thread.

Use Dex.Scheduler.spawn() to create a new fiber.

When a fiber calls one of the Dex.Future.await() functions or when it returns the fiber is suspended and execution returns to the scheduler.

By default, fibers have a 128-kb stack with a guard page at the end. Fiber stacks are pooled so that they may be reused during heavy use.

Fibers are a Dex.Future which means you can await the completion of a fiber just like any other future.

Note that fibers are pinned to a scheduler. They will not be migrated between schedulers even when a thread pool is in use.

Fiber Cancellation

Fibers may be cancelled if the fiber has been discarded by all futures awaiting completion. Fibers will always exit through a natural exit point such as a pending “await”. All attempts to await will reject with error once a fiber has been cancelled.

If you want to ignore cancellation of fibers, use Dex.Future.disown() on the fiber after creation.

This article can be found at scheduling in the libdex documentation.

Integrating Libdex and GAsyncResult

Previously.

Historically if you wanted to do asynchronous work in GObject-based applications you would use GAsyncReadyCallback and GAsyncResult.

There are two ways to integrate with this form of asynchronous API.

In one direction, you can consume this historical API and provide the result as a DexFuture. In the other direction, you can provide this API in your application or library but implement it behind the scenes with DexFuture.

Converting GAsyncResult to Futures

A typical case to integrate, at least initially, is to extract the result of a GAsyncResult and propagate it to a future.

One way to do that is with a Dex.Promise which will resolve or reject from your async callback.

static void
my_callback (GObject      *object,
             GAsyncResult *result,
             gpointer      user_data)
{
  g_autoptr(DexPromise) promise = user_data;
  g_autoptr(GError) error = NULL;

  if (thing_finish (THING (object), result, &error))
    dex_promise_resolve_boolean (promise, TRUE);
  else
    dex_promise_reject (promise, g_steal_pointer (&error));
}

DexFuture *
my_wrapper (Thing *thing)
{
  DexPromise *promise = dex_promise_new_cancellable ();

  thing_async (thing,
               dex_promise_get_cancellable (promise),
               my_callback,
               dex_ref (promise));

  return DEX_FUTURE (promise);
}

Implementing AsyncResult with Futures

In some cases you may not want to “leak” into your API that you are using DexFuture. For example, you may want to only expose a traditional GIO API or maybe even clean up legacy code.

For these cases use Dex.AsyncResult. It is designed to feel familiar to those that have used GTask.

Dex.AsyncResult.new() allows taking the typical cancellable, callback, and user_data parameters similar to GTask.

Then call Dex.AsyncResult.await() providing the future that will resolve or reject with error. One completed, the users provided callback will be executed within the active scheduler at time of creation.

From your finish function, call the appropriate propgate API such as Dex.AsyncResult.propagate_int().

void
thing_async (Thing               *thing,
             GCancellable        *cancellable,
             GAsyncReadyCallback  callback,
             gpointer             user_data)
{
  g_autoptr(DexAsyncResult) result = NULL;

  result = dex_async_result_new (thing, cancellable, callback, user_data);
  dex_async_result_await (result, dex_future_new_true ());
}

gboolean
thing_finish (Thing         *thing,
              GAsyncResult  *result,
              GError       **error)
{
  return dex_async_result_propagate_boolean (DEX_ASYNC_RESULT (result), error);
}

Safety Notes

One thing that Libdex handles better than GTask is ensuring that your user_data is destroyed on the proper thread. The design of Dex.Block was done in such a way that both the result and user_data are passed back to the owning thread at the same time. This ensures that your user_data will never be finalized on the wrong thread.

This article can be found in the documentation at Integrating GAsyncResult.

Libdex 1.0

A couple years ago I spent a great deal of time in the waiting room of an allergy clinic. So much that I finally found the time to write a library that was meant to be a followup to libgtask/libiris libraries I wrote nearly two decades ago. A lot has changed in Linux since then and I felt that maybe this time, I could get it “right”.

This will be a multi-part series, but today lets focus on terminology so we have a common language to communicate.

Futures

A future is a container that will eventually contain a result or an error.

Programmers often use the words “future” and “promise” interchangeably. Libdex tries, when possible, to follow the academic nomenclature for futures. That is to say that a future is the interface and promise is a type of future.

Futures exist in one of three states. The first state is pending. A future exists in this state until it has either rejected or resolved.

The second state is resolved. A future reaches this state when it has successfully obtained a value.

The last third state is rejected. If there was a failure to obtain a value a future will be in this state and contain a GError representing such failure.

Promises and More

A promise is a type of future that allows the creator to set the resolved value or error. This is a common type of future to use when you are integrating with things that are not yet integrated with Libdex.

Other types of futures also exist.

/* resolve to "true" */
DexPromise *good = dex_promise_new ();
dex_promise_resolve_boolean (good, TRUE);

/* reject with error */
DexPromise *bad = dex_promise_new ();
dex_promise_reject (good,
                    g_error_new (G_IO_ERROR,
                                 G_IO_ERROR_FAILED,
                                 "Failed"));

Static Futures

Sometimes you already know the result of a future upfront.
The DexStaticFuture is used in this case.
Various constructors for DexFuture will help you create one.

For example, Dex.Future.new_take_object() will create a static future for a GObject-derived instance.

DexFuture *future = dex_future_new_for_int (123);

Blocks

One of the most commonly used types of futures in Libdex is a DexBlock.

A DexBlock is a callback that is run to process the result of a future. The block itself is also a future meaning that you can chain these blocks together into robust processing groups.

“Then” Blocks

The first type of block is a “then” block which is created using Dex.Future.then(). These blocks will only be run if the dependent future resolves with a value. Otherwise, the rejection of the dependent future is propagated to the block.

static DexFuture *
further_processing (DexFuture *future,
                    gpointer   user_data)
{
  const GValue *result = dex_promise_get_value (future, NULL);

  /* since future is completed at this point, you can also use
   * the simplified "await" API. Otherwise you'd get a rejection
   * for not being on a fiber. (More on that later).
   */
  g_autoptr(GObject) object = dex_await_object (dex_ref (future), NULL);

  return dex_ref (future);
}

“Catch” Blocks

Since some futures may fail, there is value in being able to “catch” the failure and resolve it.

Use Dex.Future.catch() to handle the result of a rejected future and resolve or reject it further.

static DexFuture *
catch_rejection (DexFuture *future,
                 gpointer   user_data)
{
  g_autoptr(GError) error = NULL;

  dex_future_get_value (future, &error);

  if (g_error_matches (error, G_IO_ERROR, G_IO_ERROR_NOT_FOUND))
    return dex_future_new_true ();

  return dex_ref (future);
}

“Finally” Blocks

There may be times when you want to handle completion of a future whether it resolved or rejected. For this case, use a “finally” block by calling Dex.Future.finally().

Infinite Loops

If you find you have a case where you want a DexBlock to loop indefinitely, you can use the _loop variants of the block APIs.

See Dex.Future.then_loop(), Dex.Future.catch_loop(), or Dex.Future.finally_loop(). This is generally useful when your block’s callback will begin the next stage of work as the result of the callback.

Future Sets

A FutureSet is a type of future that is the composition of multiple futures. This is an extremely useful construct because it allows you to do work concurrently and then process the results in a sort of “reduce” phase.

For example, you could make a request to a database, cache server, and a timeout and process the first that completes.

There are multiple types of future sets based on the type of problem you want to solve.

Dex.Future.all() can be used to resolve when all dependent futures have resolved, otherwise it will reject with error once they are complete.
If you want to reject as soon as the first item rejects, Dex.Future.all_race() will get you that behavior.

Other useful Dex.FutureSet construtors include Dex.Future.any() and Dex.Future.first.

/* Either timeout or propagate result of cache/db query */
return dex_future_first (dex_timeout_new_seconds (60),
                         dex_future_any (query_db_server (),
                                         query_cache_server (),
                                         NULL),
                         NULL);

Cancellable

Many programmers who use GTK and GIO are familiar with GCancellable. Libdex has something similar in the form of DexCancellable. However, in the Libdex case, DexCancellable is a future.

It allows for convenient grouping with other futures to perform cancellation when the Dex.Cancellable.cancel() method is called.

It can also integrate with GCancellable when created using Dex.Cancellable.new_from_cancellable().

A DexCancellable will only ever reject.

DexFuture *future = dex_cancellable_new ();
dex_cancellable_cancel (DEX_CANCELLABLE (future));

Timeouts

A timeout may be represented as a future.
In this case, the timeout will reject after a time period has passed.

A DexTimeout will only ever reject.

This future is implemented ontop of GMainContext via API like g_timeout_add().

DexFuture *future = dex_timeout_new_seconds (60);

Unix Signals

Libdex can represent unix signals as a future. That is to say that the future will resolve to an integer of the signal number when that signal has been raised.

This is implemented using g_unix_signal_source_new() and comes with all the same restrictions.

Delayed

Sometimes you may run into a case where you want to gate the result of a future until a specific moment.

For this case, DexDelayed allows you to wrap another future and decide when to “uncork” the result.

DexFuture *delayed = dex_delayed_new (dex_future_new_true ());
dex_delayed_release (DEX_DELAYED (delayed));

Fibers

Another type of future is a “fiber”.

More care will be spent on fibers later on but suffice to say that the result of a fiber is easily consumable as a future via DexFiber.

DexFuture *future = dex_scheduler_spawn (NULL, 0, my_fiber, state, state_free);

Cancellation Propagation

Futures within your application will enevitably depend on other futures.

If all of the futures depending on a future have been released, the dependent future will have the opportunity to cancel itself. This allows for cascading cancellation so that unnecessary work may be elided.

You can use Dex.Future.disown() to ensure that a future will continue to be run even if the dependent futures are released.

Schedulers

Libdex requires much processing that needs to be done on the main loop of a thread. This is generally handled by a DexScheduler.

The main thread of an application has the default sheduler which is a DexMainScheduler.

Libdex also has a managed thread pool of schedulers via the DexThreadPoolScheduler.

Schedulers manage short tasks, executing DexBlock when they are ready, finalizing objects on their owning thread, and running fibers.

Schedulers integrate with the current threads GMainContext via GSource making it easy to use Libdex with GTK and Clutter-based applications.

Channels

DexChannel is a higher-level construct built on futures that allow passing work between producers and consumers. They are akin to Go channels in that they have a read and a write side. However, they are much more focused on integrating well with DexFuture.

You can find this article in the Libdex documentation under terminology.

Status Week 34

Foundry

  • Spent a bit of time working out how we can enable selection of app patterns in Foundry. The goal here would be to have some very common libadwaita usage patterns available for selection in the new-project guide.

    Ultimately it will rely on FoundryInputCombo/FoundryInputChoice but we’ll have to beef it up to support iconography.

  • Finish up a couple speed-runs so they can be uploaded to to gitlab. chergert/assist and chergert/staged are there and can serve as an example of how to use the libfoundry API.

  • A big portion of this week has been figuring out how I want to expose tweaks from libfoundry into applications. There are a lot of caveats here which make it somewhat tricky.

    For example, not every application will need every toggle, so we need a way to filter them. GListModel is probably our easiest way out with this, but we’ll likely need a bit more control over provenance of tweaks here for filtering.

  • Tweak engine has a new “path” design which allows us to dynamically query available tweaks as you dive down deeper into the UI. This is primarily to help avoid some of the slower parts of the tweaks engine in GNOME Builder.

    Also, to help support this, we can define tweaks using static data which allows for registration/query much faster. For example, there is no need to do UI merging since that can happen automatically.

    There are also cases where you may need to register tweaks which are more complex than simple GSettings. We should be able to accommodate that just fine.

  • Added API for add/remove to Git stage. Much simpler than using the libgit2 index APIs directly, for the common things.

    We may want to add a “Stage” API at some point. But for now, the helpers do the job for the non-partial-stage use-case.

    Just foundry_git_vcs_stage_entry (vcs, entry, contents) where the entry comes from your status list. You can retrieve that with foundry_git_vcs_list_status (vcs). If you are brave enough to do your own diff/patch you can implement staging lines with this too.

    However, that is probably where we want an improved stage helper.

  • Added API for creating commits.

    Much easier to do foundry_git_vcs_commit (vcs, msg, name, email) than the alternatives.

  • Running LLM tools can now be done through the conversation object which allows for better retention of state. Specifically because that can allow the conversation to track a history object other than a simple wrapped FoundryLlmMessage.

    For example, some message subclasses may have extra state like a “directory listing” which UI could use to show something more interesting than some text.

  • Simplify creating UI around FoundryLlmConversation with a new busy property that you can use to cancel your in-flight futures.

  • Fixed an issue where running the LLM tool (via subprocess) would not proxy the request back to the parent UI process. Fixing that means that you can update the application UI when the conversation is running a tool. If you were, for example, to cancel the build then the tool would get canceled too.

  • Made the spellcheck integration work the same as we do in Builder and Text Editor which is to move the cursor first on right-click before showing corrections.

Libspelling

  • New release with some small bugfixes.

Levers

  • Another speed run application which is basically the preferences part of Builder but on top of Foundry. The idea here is that you can just drop into a project and tweak most aspects of it.

    Not intended to be a “real” application like the other speed-runs, but at least it helps ensure that the API is relatively useful.

Status Week 33

This week is still largely focused on finalizing API/ABI for the 1.0 of Foundry in a few weeks.

Foundry

  • Did a bunch of work on LLM completion and and conversation APIs. They are not focused on supporting everything possible but instead making some of the common stuff extremely simple. That goes for both the model size of things and the UI side of things.

    For example, heavy usage of GListModel everywhere we can.

  • Created new abstractions for LlmTool, LlmToolProviders, and the actual call of a tool (aptly, LlmToolCall). One reason this all takes so much time to scaffold is that you want to allow some amount of flexibility when connecting models, but also avoid too much API surface area.

    I think I’ve managed to do that here.

  • Landed Ollama implementation of the FoundryLlmConversation API. The ollama server appears to be stateless, which means copying the conversation over-and-over as you go. I guess this at least gives you an idea of your context window.

  • Setup a couple tool call implementations to test out that infrastructure. For example, it’s really easy to tell the model that you build with build tool and then provide it the results.

  • Fixed some licensing issues where I mostly just forgot to update the headers when copying them over. Things should be in a good place now for distributions to adhere to their SPDX rules.

  • Language settings now have a very last resort setting which are the “defaults” we ship with the library. That is just sensible stuff like using 4 spaces for tabs/indent in Python.

    Settings at any layer can override these values.

  • Lots of work on project templates. We have both GTK 4 and Adwaita templates again. They support C/Python/rust/JavaScript like Builder does too.

    But this time I tried to go a bit further. They should have a bunch of integration bits setup which we didn’t get to before.

  • Setup an example Flatpak manifest for applications wanting to use libfoundry (see examples/flatpak/) that should help get you started.

  • Setup i18n/l10n for libfoundry. I don’t think anything is consuming translations for GNOME 49 though, so mostly just gets us up and running for 50.

  • Landed some new API for working with the stage/index within FoundryGitVcs. Tested it with a speed-run challenge a bit later on in this report.

Assist

  • To test out the LLM APIs and ensure they can actually be used I did a speed-run to implement a “Foundry-based Developer Chat” with a time limit of two hours.

    The reality is that I’m still _much_ faster writing code with all of my templates and snippets than I thought.

    The new templates in Foundry are killer though.

  • It requires a model which supports tool calls if you want to do anything interesting with it. I’m not sure if there are models which can do both written output _and_ tool-calls which makes this a bit annoying to wait while it figures out it should call a tool.

  • While doing this, I realized a bunch of little things to fix in the LLM APIs. One piece still missing that I’d want to have in the future is the ability for specialized FoundryLlmMessage which not only have text content but typed data as well.

    For example, a tool call that is essentially a ls should really display the output as an interactive directory list and not text.

    But since this was a speed run, I did not implement that. Only made sure that the APIs could adapt to it in the future.

Staged

  • Started another speed-run app to test out the version control engine we have in Foundry. This one is basically just to replace my very quick use of git-gui to line stage patches.

    Came up with a neat way to highlight old/new versions of a file and then display them with GtkListView instead of using a source view. No reason to power up the editing infrastructure if you’ll never be editing.

Manuals

  • Discovered I wasn’t getting notifications since the move to the GNOME/ namespace so flushed out the backlog of MR there.

GtkSourceView

  • Fix click-through on the overview map which broke again during this development cycle. My fault for not reviewing and/or testing better.

  • Now that we have GNOME CI doing LSAN/ASAN/UBSAN/coverage/scanbuild I went ahead and fixed a bunch of leaks that are part of the testsuite.

    Additionally, it helped me find a few that were there in everyday code use, so that is always a lovely thing to fix.

Ptyxis

  • Merge some last minute string changes before we can’t anymore.

  • Still having to unfortunately close issues which come from Debian not sourcing /etc/profile.d/vte.sh by default, thus breaking integration features.

    The good news I hear is that will be changing before long.

  • Other good news is that Ptyxis has landed in the 25.10 builds and will also be landing in Debian unstable in the near future as the default terminal.

  • After some back-and-forth I merged support for the kgx palette as the “GNOME” palette in Ptyxis. My very hopeful desire is that this becomes something maintained by the design team. The problem is just that terminal colors are a huge piles of hacks on hacks.

  • Nightly builds should be fixed. Apparently something changed in the CI setup and since we’re under chergert/ptyxis/ and not GNOME/ it didn’t get automatically applied.

  • Some styling changed in libadwaita this cycle and I needed to adapt how we propagate our styling to tab close buttons.

    Really though, this all just needs to be redone (like Text Editor and Builder) to use var() properly in CSS.

Libspelling

  • Merged patch improving life-cycle tracking of the piecetable/b+tree regions (branches/leaves).

Sysprof

  • More code review and future feature planning so we can land GSoC things after I branch for 49 (very soon I hope).

Other

  • Turned 41, saw Stevie Ray Vaughan’s broadcaster guitar, finally had the “weird” pizza at Lovely’s fifty/fifty, jammed at MoPOP with my niece.

  • Lots of random little things this week to lend a hand/ear here or there as we get closer to release.

Week 32 Status

Foundry

This week was largely around getting the new template engine landed so it can be part of the 1.0 ABI. Basically just racing to get everything landed in time to commit to the API/ABI contract.

  • FoundryTextBuffer gained some new type prerequisites to make it easier for writing applications against them. Since Foundry is a command line tool as well as a library, we don’t just use GtkTextBuffer since the CLI doesn’t even link against GTK. But it is abstracted in such a way that the GTK application would implement the FoundryTextBuffer interface with a derived GtkSourceBuffer.

  • FoundryTextSettings has landed which provides a layered approach to text editor settings similar (but better) than we have currently in GNOME Builder. There is a new modeline implementation, editorconfig, and gsettings backed settings provider which apply in that order (with per-file overrides allowed at the tip).

    Where the settings-backed implementation surpasses Builder is that it allows for layering there too. You can have user-overrides by project, project defaults, as well as Foundry defaults.

    I still need to get the default settings per-language that we have already (and are mostly shared with Text Editor too) as reasonable defaults.

  • To allow changing the GSettings-based text settings above, the foundry settings set ... command gained support for specific paths using the same :/ suffix that the gsettings command uses.

  • Spent some time on the upcoming chat API for models so I can experiment with what is possible when you control the entire tools stack.

  • Dropped some features so they wouldn’t be part of the 1.0. We can implement them later on as time permits. Specifically I don’t want to commit to a MCP or DAP implementation yet since I’m not fond of either of them as an API.

  • The FoundryInput subsystem gained support for license and language inputs. This makes it much simpler to write templates in the new internal template format.

  • Allow running foundry template create ./FILE.template to create a set of files or project from a template file. That allows you to interate on your own templates for your project without having to have them installed at the right location.

  • Wrote new project templates for empty project, shared library project, and gtk4 projects. Still need to finish the gtk4 project a bit to match feature parity with the version from Builder.

    I very much am happy with how the library project turned out because this time around it supports Gir, Pkgconfig, Vapi generation, gi-doc, and more. I still need to get library potfile support though.

    I also wrote new templates for creating gobjects and gtkwidgets in C (but we can port to other languages if necessary). This is a new type of “code template” as opposed to “project template”. It still allows for multiple files to be created in the target project.

    What is particularly useful about it though is that we can allow projects to expose templates specific to that project in the UI. In Foundry, that means you have template access to create new plugins, LSPs, and services quite easily.

  • Projects can specify their default license now to make more things just happen automatically for contributors when creating new files.

  • Templates can include the default project license header simply now by doing `{{include “license.c”}} where the suffix gets the properly commented license block.

  • The API for expand templates has changed to return a GListModel of FoundryTemplateOutput. The primary motivator here is that I want to be able to have UI in Builder that lets you preview template before actually saving the templates to disk.

  • A new API landed that we had in Builder for listing build targets. Currently, only the meson plugin implements the FoundryBuildTargetProvider. This is mostly plumbing for upcoming features.

  • The new template format is a bit of amalgamation from a few formats that is just based on my experience trying to find a way to maintain these templates.

    It starts with a GKeyFile block that describes the template and inputs to the template.

    Then you have a series of what looks like markdown code blocks. You can have conditionals around them which allows for optionally including files based on input.

    The filename for the blocks can also be expanded based on template inputs. The expansions are just TmplExpr expressions from template-glib.

    An example can be found at:

    https://gitlab.gnome.org/GNOME/foundry/-/blob/main/plugins/meson-templates/library.project

Template-GLib

  • Found some oopsies in how TmplExpr evaluated branches so fixed those up. Last year I wrote most of a C compiler and taking a look at this code really makes me want to rewrite it all. The intermixing of Yacc and GObject Introspection is ripe for improvement.

  • Added support for == and != of GStrv expressions.

Other

  • Play CI whack-a-mole for ICU changes in nightly SDKs

  • Propagate foundry changes to projects depending on it so that we have useful flatpak manifests with minimal feature flags enabled.

  • Took a look at some performance issues in GNOME OS and passed along some debugging techniques. Especially useful for when all you got is an array of registers and need to know something.

  • Libpeas release for GNOME 49 beta

Week 31 Status

Foundry

  • Added a new gutter renderer for diagnostics using the FoundryOnTypeDiagnostics described last week.

  • Write another new gutter renderer for “line changes”.

    I’m really happy with how I can use fibers w/ GWeakRef to do worker loops but not keep the “owner object” alive. As long as you have a nice way to break out of the fiber loop when the object disposes (e.g. trigger a DexCancellable/DexPromise/etc) then writing this sort of widget is cleaner/simpler than before w/ GAsyncReadyCallback.

    foundry-changes-gutter-renderer.c

  • Added a :show-overview property to the line changes renderer which conveniently allows it to work as both a per-line change status and be placed in the right-side gutter as an overview of the whole document to see your place in it. Builder just recently got this feature implemented by Nokse and this is basically just a simplified version of that thanks to fibers.

  • Abstract TTY auth input into a new FoundryInput abstraction. This is currently used by the git subsystem to acquire credentials for SSH, krb, user, user/pass, etc depending on what the peer supports. However, it became pretty obvious to me that we can use it for more than just Git. It maps pretty well to at least two more features coming down the pipeline.

    Since the input mechanisms are used on a thread for TTY input (to avoid blocking main loops, fiber schedulers, etc), they needed to be thread-safe. Most things are immutable and a few well controlled places are mutable.

    The concept of a validator is implemented externally as a FoundryInputValidator which allows for re-use and separating the mechanism from policy. Quite like how it turned out honestly.

    There are abstractions for text, switches, choices, files. You might notice they will map fairly well to AdwPreferenceRow things and that is by design, since in the apps I manage, that would be their intended display mechanism.

  • Templates have finally landed in Foundry with the introduction of a FoundryTemplateManager, FoundryTemplateProvider, and FoundryTemplate. They use the new generalized FoundryInput abstractions that were discussed above.

    That allows for a foundry template list command to list templates and foundry template create to expand a certain template.

    The FoundryInput of the templates are queried via the PTY just like username/password auth works via FoundryInput. Questions are asked, input received, template expansion may continue.

    This will also allow for dynamic creation of the “Create Template” widgetry in Builder later on without sacrificing on design.

  • Meson templates from Builder have also been ported over which means that you can actually use those foundry template commands above to replace your use of Builder if that is all you used it for.

    All the normal ones are there (GTK, Adwaita, library, cli, etc).

  • A new license abstraction was created so that libraries and tooling can get access to licenses/snippets in a simple form w/o duplication. That generally gets used for template expansion and file headers.

  • The FoundryBuildPipeline gained a new vfunc for prepare_to_run(). We always had this in Builder but it never came over to Foundry until now.

    This is the core mechanism behind being able to run a command as if it were the target application (e.g. unit tests).

  • After doing the template work, I realized that we should probably just auto initialize the project so you don’t have to run foundry init afterwards. Extracted the mechanism for setting up the initial .foundry directory state and made templates use that.

  • One of the build pipeline mechanisms still missing from Builder is the ability to sit in the middle of a PTY and extract build diagnostics. This is how errors from GCC are extracted during the build (as well as for other languages).

    So I brought over our “PTY intercept” which takes your consumer FD and creates a producer FD which is bridged to another consumer FD.

    Then the JIT’d error extract regexes may be run over the middle and then create diagnostics as necessary.

    To make this simple to consume in applications, a new FoundryPtyDiagnostics object is created. You set the PTY to use for that and attach it’s intercept PTY to the build/run managers default PTY and then all the GAction will wire up correctly. That object is also a GListModel making it easy to display in application UI.

  • A FoundryService is managed by the FoundryContext. They are just subsystems that combine to useful things in Foundry. One way they can be interacted with is GAction as the base class implements GActionGroup.

    I did some cleanup to make this work well and now you can just attach the FoundryContexts GActionGroup using foundry_context_dup_action_group() to a GtkWindow using gtk_widget_insert_action_group(). At that point your buttons are basically just "context.build-manager.build" for the action-name property.

    All sorts of services export actions now for operations like build, run, clean, invalidate, purge, update dependencies, etc.

    There is a test GTK app in testsuite/tools/ you can play with this all to get ideas and/or integrate into your own app. It also integrates the live diagnostics/PTY code to exemplify that.

  • Fixed the FoundryNoRun tool to connect to the proper PTY in the deployment/run phase.

  • The purge operation now writes information about what files are being deleted to the default build PTY.

  • The new FoundryTextSettings abstraction has landed which is roughly similar to IdeFileSettings in Builder. This time it is much cleaned up now that we have DexFuture to work with.

    I’ve ported the editorconfig support over to use this as well as a new implementation of modeline support which again, is a lot simpler now that we can use fibers/threadpools effectively.

    Plugins can set their text-settings priority in their .plugin file. That way settings can have a specific order such as user-overrides, modelines, editorconfig, gsettings overrides, language defaults, and what-not.

  • The FoundryVcs gained a new foundry_vcs_query_file_status() API which allows querying for the, shocking, file status. That will give you bitflags to know in both the stage or working tree if a file is new/modified/deleted.

    To make this even more useful, you can use the FoundryDirectoryListing class (which is a GListModel of FoundryDirectoryItem) to include vcs::status file-attribute and your GFileInfo will be populated with the uint32 bitflags for a key under the same name.

    It’s also provided as a property on the FoundryDirectoryItem to make writing those git “status icons” dead simple in file panels.

Boxes

  • Found an issue w/ trailing \x00 in paths when new Boxes is opening an ISO from disk on a system with older xdg portals. Sent a pointer on the issue tracker to what Text Editor had to do as well here.

Libpeas

  • GJS gained support for pkgconfig variables and we use that now to determine which mozjs version to link against. That is required to be able to use the proper JS API we need to setup the context.

Ptyxis

  • Merged some improves to the custom link support in Ptyxis. This is used to allow you to highlight custom URL regexes. So you can turn things like “RHEL-1234” into a link to the RHEL issue tracker.

  • Track down an issue filed about titles not updating tab/window titles. It was just an issue with $PROMPT_COMMAND overwriting what they had just changed.

Text Editor

  • A lot of the maintainership of this program is just directing people to the right place. Be that GtkSourceView, GTK, shared-mime-info, etc. Do more of that.

    As an aside, I really wish people spent more time understanding how things work rather than fire-and-forget. The FOSS community used to take pride in ensuring the issue reports landed in the right place to avoid over burdening maintainers, and I’m sad that is been lost in the past decade or so. Probably just a sign of success.

Builder

  • Did a quick and dirty fix for a hang that could slow down startup due to the Manuals code going to the worker process to get the default architecture. Builder doesn’t link against Flatpak in the UI process hence why that did it. But it’s also super easy to put a couple line hard-coded #ifdef and avoid the whole RPC.

Libdex

  • Released 0.11.1 for GNOME 49.beta. I’m strongly considering making the actual 49 release our 1.0. Things have really solidified over the past year with libdex and I’m happy enough to put my stamp of approval on that.

Libspelling

  • Fix an issue with discovery of the no-spellcheck-tag which is used to avoid spellchecking things that are general syntax in language specifications. Helps a bunch when loading a large document and that can get out of sync/changed before the worker discovers it.

  • Fixed a LSAN discovered leak in the testsuite. Still one more to go. Fought LSAN and CI a bit because I can’t seem to reproduce what the CI systems get.

Other

  • Told Chat-GPT to spit me out a throw away script that parses my status reports and converts them into something generally usable by WordPress. Obviously there is a lot of dislike/scrutiny/distrust of LLMs and their creators/operators, but I really don’t see the metaphorical cat going back in the bag when you enable people in a few seconds to scratch an itch. I certainly hope we continue to scrutinize and control scope though.