Libdex 1.0

A couple years ago I spent a great deal of time in the waiting room of an allergy clinic. So much that I finally found the time to write a library that was meant to be a followup to libgtask/libiris libraries I wrote nearly two decades ago. A lot has changed in Linux since then and I felt that maybe this time, I could get it “right”.

This will be a multi-part series, but today lets focus on terminology so we have a common language to communicate.

Futures

A future is a container that will eventually contain a result or an error.

Programmers often use the words “future” and “promise” interchangeably. Libdex tries, when possible, to follow the academic nomenclature for futures. That is to say that a future is the interface and promise is a type of future.

Futures exist in one of three states. The first state is pending. A future exists in this state until it has either rejected or resolved.

The second state is resolved. A future reaches this state when it has successfully obtained a value.

The last third state is rejected. If there was a failure to obtain a value a future will be in this state and contain a GError representing such failure.

Promises and More

A promise is a type of future that allows the creator to set the resolved value or error. This is a common type of future to use when you are integrating with things that are not yet integrated with Libdex.

Other types of futures also exist.

/* resolve to "true" */
DexPromise *good = dex_promise_new ();
dex_promise_resolve_boolean (good, TRUE);

/* reject with error */
DexPromise *bad = dex_promise_new ();
dex_promise_reject (good,
                    g_error_new (G_IO_ERROR,
                                 G_IO_ERROR_FAILED,
                                 "Failed"));

Static Futures

Sometimes you already know the result of a future upfront.
The DexStaticFuture is used in this case.
Various constructors for DexFuture will help you create one.

For example, Dex.Future.new_take_object() will create a static future for a GObject-derived instance.

DexFuture *future = dex_future_new_for_int (123);

Blocks

One of the most commonly used types of futures in Libdex is a DexBlock.

A DexBlock is a callback that is run to process the result of a future. The block itself is also a future meaning that you can chain these blocks together into robust processing groups.

“Then” Blocks

The first type of block is a “then” block which is created using Dex.Future.then(). These blocks will only be run if the dependent future resolves with a value. Otherwise, the rejection of the dependent future is propagated to the block.

static DexFuture *
further_processing (DexFuture *future,
                    gpointer   user_data)
{
  const GValue *result = dex_promise_get_value (future, NULL);

  /* since future is completed at this point, you can also use
   * the simplified "await" API. Otherwise you'd get a rejection
   * for not being on a fiber. (More on that later).
   */
  g_autoptr(GObject) object = dex_await_object (dex_ref (future), NULL);

  return dex_ref (future);
}

“Catch” Blocks

Since some futures may fail, there is value in being able to “catch” the failure and resolve it.

Use Dex.Future.catch() to handle the result of a rejected future and resolve or reject it further.

static DexFuture *
catch_rejection (DexFuture *future,
                 gpointer   user_data)
{
  g_autoptr(GError) error = NULL;

  dex_future_get_value (future, &error);

  if (g_error_matches (error, G_IO_ERROR, G_IO_ERROR_NOT_FOUND))
    return dex_future_new_true ();

  return dex_ref (future);
}

“Finally” Blocks

There may be times when you want to handle completion of a future whether it resolved or rejected. For this case, use a “finally” block by calling Dex.Future.finally().

Infinite Loops

If you find you have a case where you want a DexBlock to loop indefinitely, you can use the _loop variants of the block APIs.

See Dex.Future.then_loop(), Dex.Future.catch_loop(), or Dex.Future.finally_loop(). This is generally useful when your block’s callback will begin the next stage of work as the result of the callback.

Future Sets

A FutureSet is a type of future that is the composition of multiple futures. This is an extremely useful construct because it allows you to do work concurrently and then process the results in a sort of “reduce” phase.

For example, you could make a request to a database, cache server, and a timeout and process the first that completes.

There are multiple types of future sets based on the type of problem you want to solve.

Dex.Future.all() can be used to resolve when all dependent futures have resolved, otherwise it will reject with error once they are complete.
If you want to reject as soon as the first item rejects, Dex.Future.all_race() will get you that behavior.

Other useful Dex.FutureSet construtors include Dex.Future.any() and Dex.Future.first.

/* Either timeout or propagate result of cache/db query */
return dex_future_first (dex_timeout_new_seconds (60),
                         dex_future_any (query_db_server (),
                                         query_cache_server (),
                                         NULL),
                         NULL);

Cancellable

Many programmers who use GTK and GIO are familiar with GCancellable. Libdex has something similar in the form of DexCancellable. However, in the Libdex case, DexCancellable is a future.

It allows for convenient grouping with other futures to perform cancellation when the Dex.Cancellable.cancel() method is called.

It can also integrate with GCancellable when created using Dex.Cancellable.new_from_cancellable().

A DexCancellable will only ever reject.

DexFuture *future = dex_cancellable_new ();
dex_cancellable_cancel (DEX_CANCELLABLE (future));

Timeouts

A timeout may be represented as a future.
In this case, the timeout will reject after a time period has passed.

A DexTimeout will only ever reject.

This future is implemented ontop of GMainContext via API like g_timeout_add().

DexFuture *future = dex_timeout_new_seconds (60);

Unix Signals

Libdex can represent unix signals as a future. That is to say that the future will resolve to an integer of the signal number when that signal has been raised.

This is implemented using g_unix_signal_source_new() and comes with all the same restrictions.

Delayed

Sometimes you may run into a case where you want to gate the result of a future until a specific moment.

For this case, DexDelayed allows you to wrap another future and decide when to “uncork” the result.

DexFuture *delayed = dex_delayed_new (dex_future_new_true ());
dex_delayed_release (DEX_DELAYED (delayed));

Fibers

Another type of future is a “fiber”.

More care will be spent on fibers later on but suffice to say that the result of a fiber is easily consumable as a future via DexFiber.

DexFuture *future = dex_scheduler_spawn (NULL, 0, my_fiber, state, state_free);

Cancellation Propagation

Futures within your application will enevitably depend on other futures.

If all of the futures depending on a future have been released, the dependent future will have the opportunity to cancel itself. This allows for cascading cancellation so that unnecessary work may be elided.

You can use Dex.Future.disown() to ensure that a future will continue to be run even if the dependent futures are released.

Schedulers

Libdex requires much processing that needs to be done on the main loop of a thread. This is generally handled by a DexScheduler.

The main thread of an application has the default sheduler which is a DexMainScheduler.

Libdex also has a managed thread pool of schedulers via the DexThreadPoolScheduler.

Schedulers manage short tasks, executing DexBlock when they are ready, finalizing objects on their owning thread, and running fibers.

Schedulers integrate with the current threads GMainContext via GSource making it easy to use Libdex with GTK and Clutter-based applications.

Channels

DexChannel is a higher-level construct built on futures that allow passing work between producers and consumers. They are akin to Go channels in that they have a read and a write side. However, they are much more focused on integrating well with DexFuture.

You can find this article in the Libdex documentation under terminology.

Status Week 34

Foundry

  • Spent a bit of time working out how we can enable selection of app patterns in Foundry. The goal here would be to have some very common libadwaita usage patterns available for selection in the new-project guide.

    Ultimately it will rely on FoundryInputCombo/FoundryInputChoice but we’ll have to beef it up to support iconography.

  • Finish up a couple speed-runs so they can be uploaded to to gitlab. chergert/assist and chergert/staged are there and can serve as an example of how to use the libfoundry API.

  • A big portion of this week has been figuring out how I want to expose tweaks from libfoundry into applications. There are a lot of caveats here which make it somewhat tricky.

    For example, not every application will need every toggle, so we need a way to filter them. GListModel is probably our easiest way out with this, but we’ll likely need a bit more control over provenance of tweaks here for filtering.

  • Tweak engine has a new “path” design which allows us to dynamically query available tweaks as you dive down deeper into the UI. This is primarily to help avoid some of the slower parts of the tweaks engine in GNOME Builder.

    Also, to help support this, we can define tweaks using static data which allows for registration/query much faster. For example, there is no need to do UI merging since that can happen automatically.

    There are also cases where you may need to register tweaks which are more complex than simple GSettings. We should be able to accommodate that just fine.

  • Added API for add/remove to Git stage. Much simpler than using the libgit2 index APIs directly, for the common things.

    We may want to add a “Stage” API at some point. But for now, the helpers do the job for the non-partial-stage use-case.

    Just foundry_git_vcs_stage_entry (vcs, entry, contents) where the entry comes from your status list. You can retrieve that with foundry_git_vcs_list_status (vcs). If you are brave enough to do your own diff/patch you can implement staging lines with this too.

    However, that is probably where we want an improved stage helper.

  • Added API for creating commits.

    Much easier to do foundry_git_vcs_commit (vcs, msg, name, email) than the alternatives.

  • Running LLM tools can now be done through the conversation object which allows for better retention of state. Specifically because that can allow the conversation to track a history object other than a simple wrapped FoundryLlmMessage.

    For example, some message subclasses may have extra state like a “directory listing” which UI could use to show something more interesting than some text.

  • Simplify creating UI around FoundryLlmConversation with a new busy property that you can use to cancel your in-flight futures.

  • Fixed an issue where running the LLM tool (via subprocess) would not proxy the request back to the parent UI process. Fixing that means that you can update the application UI when the conversation is running a tool. If you were, for example, to cancel the build then the tool would get canceled too.

  • Made the spellcheck integration work the same as we do in Builder and Text Editor which is to move the cursor first on right-click before showing corrections.

Libspelling

  • New release with some small bugfixes.

Levers

  • Another speed run application which is basically the preferences part of Builder but on top of Foundry. The idea here is that you can just drop into a project and tweak most aspects of it.

    Not intended to be a “real” application like the other speed-runs, but at least it helps ensure that the API is relatively useful.

Status Week 33

This week is still largely focused on finalizing API/ABI for the 1.0 of Foundry in a few weeks.

Foundry

  • Did a bunch of work on LLM completion and and conversation APIs. They are not focused on supporting everything possible but instead making some of the common stuff extremely simple. That goes for both the model size of things and the UI side of things.

    For example, heavy usage of GListModel everywhere we can.

  • Created new abstractions for LlmTool, LlmToolProviders, and the actual call of a tool (aptly, LlmToolCall). One reason this all takes so much time to scaffold is that you want to allow some amount of flexibility when connecting models, but also avoid too much API surface area.

    I think I’ve managed to do that here.

  • Landed Ollama implementation of the FoundryLlmConversation API. The ollama server appears to be stateless, which means copying the conversation over-and-over as you go. I guess this at least gives you an idea of your context window.

  • Setup a couple tool call implementations to test out that infrastructure. For example, it’s really easy to tell the model that you build with build tool and then provide it the results.

  • Fixed some licensing issues where I mostly just forgot to update the headers when copying them over. Things should be in a good place now for distributions to adhere to their SPDX rules.

  • Language settings now have a very last resort setting which are the “defaults” we ship with the library. That is just sensible stuff like using 4 spaces for tabs/indent in Python.

    Settings at any layer can override these values.

  • Lots of work on project templates. We have both GTK 4 and Adwaita templates again. They support C/Python/rust/JavaScript like Builder does too.

    But this time I tried to go a bit further. They should have a bunch of integration bits setup which we didn’t get to before.

  • Setup an example Flatpak manifest for applications wanting to use libfoundry (see examples/flatpak/) that should help get you started.

  • Setup i18n/l10n for libfoundry. I don’t think anything is consuming translations for GNOME 49 though, so mostly just gets us up and running for 50.

  • Landed some new API for working with the stage/index within FoundryGitVcs. Tested it with a speed-run challenge a bit later on in this report.

Assist

  • To test out the LLM APIs and ensure they can actually be used I did a speed-run to implement a “Foundry-based Developer Chat” with a time limit of two hours.

    The reality is that I’m still _much_ faster writing code with all of my templates and snippets than I thought.

    The new templates in Foundry are killer though.

  • It requires a model which supports tool calls if you want to do anything interesting with it. I’m not sure if there are models which can do both written output _and_ tool-calls which makes this a bit annoying to wait while it figures out it should call a tool.

  • While doing this, I realized a bunch of little things to fix in the LLM APIs. One piece still missing that I’d want to have in the future is the ability for specialized FoundryLlmMessage which not only have text content but typed data as well.

    For example, a tool call that is essentially a ls should really display the output as an interactive directory list and not text.

    But since this was a speed run, I did not implement that. Only made sure that the APIs could adapt to it in the future.

Staged

  • Started another speed-run app to test out the version control engine we have in Foundry. This one is basically just to replace my very quick use of git-gui to line stage patches.

    Came up with a neat way to highlight old/new versions of a file and then display them with GtkListView instead of using a source view. No reason to power up the editing infrastructure if you’ll never be editing.

Manuals

  • Discovered I wasn’t getting notifications since the move to the GNOME/ namespace so flushed out the backlog of MR there.

GtkSourceView

  • Fix click-through on the overview map which broke again during this development cycle. My fault for not reviewing and/or testing better.

  • Now that we have GNOME CI doing LSAN/ASAN/UBSAN/coverage/scanbuild I went ahead and fixed a bunch of leaks that are part of the testsuite.

    Additionally, it helped me find a few that were there in everyday code use, so that is always a lovely thing to fix.

Ptyxis

  • Merge some last minute string changes before we can’t anymore.

  • Still having to unfortunately close issues which come from Debian not sourcing /etc/profile.d/vte.sh by default, thus breaking integration features.

    The good news I hear is that will be changing before long.

  • Other good news is that Ptyxis has landed in the 25.10 builds and will also be landing in Debian unstable in the near future as the default terminal.

  • After some back-and-forth I merged support for the kgx palette as the “GNOME” palette in Ptyxis. My very hopeful desire is that this becomes something maintained by the design team. The problem is just that terminal colors are a huge piles of hacks on hacks.

  • Nightly builds should be fixed. Apparently something changed in the CI setup and since we’re under chergert/ptyxis/ and not GNOME/ it didn’t get automatically applied.

  • Some styling changed in libadwaita this cycle and I needed to adapt how we propagate our styling to tab close buttons.

    Really though, this all just needs to be redone (like Text Editor and Builder) to use var() properly in CSS.

Libspelling

  • Merged patch improving life-cycle tracking of the piecetable/b+tree regions (branches/leaves).

Sysprof

  • More code review and future feature planning so we can land GSoC things after I branch for 49 (very soon I hope).

Other

  • Turned 41, saw Stevie Ray Vaughan’s broadcaster guitar, finally had the “weird” pizza at Lovely’s fifty/fifty, jammed at MoPOP with my niece.

  • Lots of random little things this week to lend a hand/ear here or there as we get closer to release.

Week 32 Status

Foundry

This week was largely around getting the new template engine landed so it can be part of the 1.0 ABI. Basically just racing to get everything landed in time to commit to the API/ABI contract.

  • FoundryTextBuffer gained some new type prerequisites to make it easier for writing applications against them. Since Foundry is a command line tool as well as a library, we don’t just use GtkTextBuffer since the CLI doesn’t even link against GTK. But it is abstracted in such a way that the GTK application would implement the FoundryTextBuffer interface with a derived GtkSourceBuffer.

  • FoundryTextSettings has landed which provides a layered approach to text editor settings similar (but better) than we have currently in GNOME Builder. There is a new modeline implementation, editorconfig, and gsettings backed settings provider which apply in that order (with per-file overrides allowed at the tip).

    Where the settings-backed implementation surpasses Builder is that it allows for layering there too. You can have user-overrides by project, project defaults, as well as Foundry defaults.

    I still need to get the default settings per-language that we have already (and are mostly shared with Text Editor too) as reasonable defaults.

  • To allow changing the GSettings-based text settings above, the foundry settings set ... command gained support for specific paths using the same :/ suffix that the gsettings command uses.

  • Spent some time on the upcoming chat API for models so I can experiment with what is possible when you control the entire tools stack.

  • Dropped some features so they wouldn’t be part of the 1.0. We can implement them later on as time permits. Specifically I don’t want to commit to a MCP or DAP implementation yet since I’m not fond of either of them as an API.

  • The FoundryInput subsystem gained support for license and language inputs. This makes it much simpler to write templates in the new internal template format.

  • Allow running foundry template create ./FILE.template to create a set of files or project from a template file. That allows you to interate on your own templates for your project without having to have them installed at the right location.

  • Wrote new project templates for empty project, shared library project, and gtk4 projects. Still need to finish the gtk4 project a bit to match feature parity with the version from Builder.

    I very much am happy with how the library project turned out because this time around it supports Gir, Pkgconfig, Vapi generation, gi-doc, and more. I still need to get library potfile support though.

    I also wrote new templates for creating gobjects and gtkwidgets in C (but we can port to other languages if necessary). This is a new type of “code template” as opposed to “project template”. It still allows for multiple files to be created in the target project.

    What is particularly useful about it though is that we can allow projects to expose templates specific to that project in the UI. In Foundry, that means you have template access to create new plugins, LSPs, and services quite easily.

  • Projects can specify their default license now to make more things just happen automatically for contributors when creating new files.

  • Templates can include the default project license header simply now by doing `{{include “license.c”}} where the suffix gets the properly commented license block.

  • The API for expand templates has changed to return a GListModel of FoundryTemplateOutput. The primary motivator here is that I want to be able to have UI in Builder that lets you preview template before actually saving the templates to disk.

  • A new API landed that we had in Builder for listing build targets. Currently, only the meson plugin implements the FoundryBuildTargetProvider. This is mostly plumbing for upcoming features.

  • The new template format is a bit of amalgamation from a few formats that is just based on my experience trying to find a way to maintain these templates.

    It starts with a GKeyFile block that describes the template and inputs to the template.

    Then you have a series of what looks like markdown code blocks. You can have conditionals around them which allows for optionally including files based on input.

    The filename for the blocks can also be expanded based on template inputs. The expansions are just TmplExpr expressions from template-glib.

    An example can be found at:

    https://gitlab.gnome.org/GNOME/foundry/-/blob/main/plugins/meson-templates/library.project

Template-GLib

  • Found some oopsies in how TmplExpr evaluated branches so fixed those up. Last year I wrote most of a C compiler and taking a look at this code really makes me want to rewrite it all. The intermixing of Yacc and GObject Introspection is ripe for improvement.

  • Added support for == and != of GStrv expressions.

Other

  • Play CI whack-a-mole for ICU changes in nightly SDKs

  • Propagate foundry changes to projects depending on it so that we have useful flatpak manifests with minimal feature flags enabled.

  • Took a look at some performance issues in GNOME OS and passed along some debugging techniques. Especially useful for when all you got is an array of registers and need to know something.

  • Libpeas release for GNOME 49 beta

Week 31 Status

Foundry

  • Added a new gutter renderer for diagnostics using the FoundryOnTypeDiagnostics described last week.

  • Write another new gutter renderer for “line changes”.

    I’m really happy with how I can use fibers w/ GWeakRef to do worker loops but not keep the “owner object” alive. As long as you have a nice way to break out of the fiber loop when the object disposes (e.g. trigger a DexCancellable/DexPromise/etc) then writing this sort of widget is cleaner/simpler than before w/ GAsyncReadyCallback.

    foundry-changes-gutter-renderer.c

  • Added a :show-overview property to the line changes renderer which conveniently allows it to work as both a per-line change status and be placed in the right-side gutter as an overview of the whole document to see your place in it. Builder just recently got this feature implemented by Nokse and this is basically just a simplified version of that thanks to fibers.

  • Abstract TTY auth input into a new FoundryInput abstraction. This is currently used by the git subsystem to acquire credentials for SSH, krb, user, user/pass, etc depending on what the peer supports. However, it became pretty obvious to me that we can use it for more than just Git. It maps pretty well to at least two more features coming down the pipeline.

    Since the input mechanisms are used on a thread for TTY input (to avoid blocking main loops, fiber schedulers, etc), they needed to be thread-safe. Most things are immutable and a few well controlled places are mutable.

    The concept of a validator is implemented externally as a FoundryInputValidator which allows for re-use and separating the mechanism from policy. Quite like how it turned out honestly.

    There are abstractions for text, switches, choices, files. You might notice they will map fairly well to AdwPreferenceRow things and that is by design, since in the apps I manage, that would be their intended display mechanism.

  • Templates have finally landed in Foundry with the introduction of a FoundryTemplateManager, FoundryTemplateProvider, and FoundryTemplate. They use the new generalized FoundryInput abstractions that were discussed above.

    That allows for a foundry template list command to list templates and foundry template create to expand a certain template.

    The FoundryInput of the templates are queried via the PTY just like username/password auth works via FoundryInput. Questions are asked, input received, template expansion may continue.

    This will also allow for dynamic creation of the “Create Template” widgetry in Builder later on without sacrificing on design.

  • Meson templates from Builder have also been ported over which means that you can actually use those foundry template commands above to replace your use of Builder if that is all you used it for.

    All the normal ones are there (GTK, Adwaita, library, cli, etc).

  • A new license abstraction was created so that libraries and tooling can get access to licenses/snippets in a simple form w/o duplication. That generally gets used for template expansion and file headers.

  • The FoundryBuildPipeline gained a new vfunc for prepare_to_run(). We always had this in Builder but it never came over to Foundry until now.

    This is the core mechanism behind being able to run a command as if it were the target application (e.g. unit tests).

  • After doing the template work, I realized that we should probably just auto initialize the project so you don’t have to run foundry init afterwards. Extracted the mechanism for setting up the initial .foundry directory state and made templates use that.

  • One of the build pipeline mechanisms still missing from Builder is the ability to sit in the middle of a PTY and extract build diagnostics. This is how errors from GCC are extracted during the build (as well as for other languages).

    So I brought over our “PTY intercept” which takes your consumer FD and creates a producer FD which is bridged to another consumer FD.

    Then the JIT’d error extract regexes may be run over the middle and then create diagnostics as necessary.

    To make this simple to consume in applications, a new FoundryPtyDiagnostics object is created. You set the PTY to use for that and attach it’s intercept PTY to the build/run managers default PTY and then all the GAction will wire up correctly. That object is also a GListModel making it easy to display in application UI.

  • A FoundryService is managed by the FoundryContext. They are just subsystems that combine to useful things in Foundry. One way they can be interacted with is GAction as the base class implements GActionGroup.

    I did some cleanup to make this work well and now you can just attach the FoundryContexts GActionGroup using foundry_context_dup_action_group() to a GtkWindow using gtk_widget_insert_action_group(). At that point your buttons are basically just "context.build-manager.build" for the action-name property.

    All sorts of services export actions now for operations like build, run, clean, invalidate, purge, update dependencies, etc.

    There is a test GTK app in testsuite/tools/ you can play with this all to get ideas and/or integrate into your own app. It also integrates the live diagnostics/PTY code to exemplify that.

  • Fixed the FoundryNoRun tool to connect to the proper PTY in the deployment/run phase.

  • The purge operation now writes information about what files are being deleted to the default build PTY.

  • The new FoundryTextSettings abstraction has landed which is roughly similar to IdeFileSettings in Builder. This time it is much cleaned up now that we have DexFuture to work with.

    I’ve ported the editorconfig support over to use this as well as a new implementation of modeline support which again, is a lot simpler now that we can use fibers/threadpools effectively.

    Plugins can set their text-settings priority in their .plugin file. That way settings can have a specific order such as user-overrides, modelines, editorconfig, gsettings overrides, language defaults, and what-not.

  • The FoundryVcs gained a new foundry_vcs_query_file_status() API which allows querying for the, shocking, file status. That will give you bitflags to know in both the stage or working tree if a file is new/modified/deleted.

    To make this even more useful, you can use the FoundryDirectoryListing class (which is a GListModel of FoundryDirectoryItem) to include vcs::status file-attribute and your GFileInfo will be populated with the uint32 bitflags for a key under the same name.

    It’s also provided as a property on the FoundryDirectoryItem to make writing those git “status icons” dead simple in file panels.

Boxes

  • Found an issue w/ trailing \x00 in paths when new Boxes is opening an ISO from disk on a system with older xdg portals. Sent a pointer on the issue tracker to what Text Editor had to do as well here.

Libpeas

  • GJS gained support for pkgconfig variables and we use that now to determine which mozjs version to link against. That is required to be able to use the proper JS API we need to setup the context.

Ptyxis

  • Merged some improves to the custom link support in Ptyxis. This is used to allow you to highlight custom URL regexes. So you can turn things like “RHEL-1234” into a link to the RHEL issue tracker.

  • Track down an issue filed about titles not updating tab/window titles. It was just an issue with $PROMPT_COMMAND overwriting what they had just changed.

Text Editor

  • A lot of the maintainership of this program is just directing people to the right place. Be that GtkSourceView, GTK, shared-mime-info, etc. Do more of that.

    As an aside, I really wish people spent more time understanding how things work rather than fire-and-forget. The FOSS community used to take pride in ensuring the issue reports landed in the right place to avoid over burdening maintainers, and I’m sad that is been lost in the past decade or so. Probably just a sign of success.

Builder

  • Did a quick and dirty fix for a hang that could slow down startup due to the Manuals code going to the worker process to get the default architecture. Builder doesn’t link against Flatpak in the UI process hence why that did it. But it’s also super easy to put a couple line hard-coded #ifdef and avoid the whole RPC.

Libdex

  • Released 0.11.1 for GNOME 49.beta. I’m strongly considering making the actual 49 release our 1.0. Things have really solidified over the past year with libdex and I’m happy enough to put my stamp of approval on that.

Libspelling

  • Fix an issue with discovery of the no-spellcheck-tag which is used to avoid spellchecking things that are general syntax in language specifications. Helps a bunch when loading a large document and that can get out of sync/changed before the worker discovers it.

  • Fixed a LSAN discovered leak in the testsuite. Still one more to go. Fought LSAN and CI a bit because I can’t seem to reproduce what the CI systems get.

Other

  • Told Chat-GPT to spit me out a throw away script that parses my status reports and converts them into something generally usable by WordPress. Obviously there is a lot of dislike/scrutiny/distrust of LLMs and their creators/operators, but I really don’t see the metaphorical cat going back in the bag when you enable people in a few seconds to scratch an itch. I certainly hope we continue to scrutinize and control scope though.

Week 30 Status

My approach to engineering involves an engineers notebook and pen at my side almost all the time. My ADHD is so bad that without writing things down I would very much not remember what I did.

Working at large companies can have a silencing effect on engineers in the community because all our communication energy is burnt on weekly status reports. You see this all the time, and it was famously expected behavior when FOSS people joined Google.

But it is not unique to Google and I certainly suffer from it myself. So I’m going to try to experiment for a while dropping my status reports here too, at least for the things that aren’t extremely specific to my employer.

Open Questions

  • What is the state-of-the-art right now for “I want to provide a completely artisan file-system to a container”. For example, say I wanted to have a FUSE file-system for that build pipeline or other tooling accessed.

    At least when it comes to project sources. Everything else should be read-only anyway.

    It would be nice to allow tooling some read/write access but gate the writes so they are limited to the tools running and not persistent when the tool returns.

Foundry

  • A bit more testing of Foundry’s replacement for Jsonrpc-GLib, which is a new libdex based FoundryJsonrpcDriver. It knows how to talk a few different types (HTTP-style, \0 or \n delimited, etc).

    LSP backend has been ported to this now along with all the JSON node creating helpers so try to put those through their paces.

  • Add pre-load/post-load to FoundryTextDocumentAddin so that we can add hooks for addins early in the loading process. We actually want this more for avoiding things during buffer loading.

  • Found a nasty issue where creating addins was causing long running leaks do to the GParameter arrays getting saved for future addin creation. Need to be a bit more clever about initial property setup so that we don’t create this reference cycle.

  • New word-completion plugin for Foundry that takes a different approach from what we did in GtkSourceView. Instead, this runs on demand with a copy of the document buffer on a fiber on a thread. This allows using regex for word boundaries (\w) with JIT, no synchronization with GTK, and just generally _a lot_ faster. It also allowed for following referenced files from #include style style headers in C/C++/Obj-C which is something VIM does (at least with plugins) that I very much wanted.

    It is nice knowing when a symbol comes from the local file vs an included file as well (again, VIM does this) so I implemented that as well for completeness.

    Make sure it does word de-duplication while I’m at it.

  • Preserve completion activation (user initialized, automatic, etc) to propagate to the completion providers.

  • Live diagnostics tracking is much easier now. You can just create a FoundryOnTypeDiagnostics(document) and it will manage updating things as you go. It is also smart enough to do this with GWeakRef so that we don’t keep underlying buffers/documents/etc alive past the point they should be unloaded (as the worker runs on a fiber).

    You can share a single instance of the live diagnostics using foundry_text_document_watch_diagnostics() to avoid extra work.

  • Add a Git-specific clone API in FoundryGitClone which handles all the annoying things like SSH authentication/etc via the use of our prompt abstraction (TTY, app dialog, etc). This also means there is a new foundry clone ... CLI command to test that infrastructure outside of the IDE. Should help for tracking down weird integration issues.

  • To make the Git cloner API work well I had to remove the context requirement from FoundryAuthPrompt. You’ll never have a loaded context when you want to clone (as there is not yet a project) so that requirement was nonsensical.

  • Add new foundry_vcs_list_commits_with_file() API to get the commit history on a single file. This gives you a list model of FoundryCommit which should make it very easy for applications to browse through file history. One call, bridge the model to a listview and wire up some labels.

  • Add FoundryVcsTree, FoundryVcsDiff, FoundryVcsDelta types and git implementations of them. Like the rest of the new Git abstractions, this all runs threaded using libdex and futures which complete when the thread returns. Still need to iterate on this a bit before the 1.0 API is finalized.

  • New API to generate diffs from trees or find trees by identifiers.

  • Found out that libgit2 does not support the bitmap index of the command line git command. That means that you have to do a lot of diffing to determine what commits contain a specific file. Maybe that will change in the future though. We could always shell out to the git command for this specific operation if it ends up too slow.

  • New CTags parser that allows for read-only memory. Instead of doing the optimization in the past (insert \0 and use strings in place) the new index keeps string offset/run for a few important parts.

    Then the open-coded binary search to find the nearest partial match against (then walking backward to get first potential match) can keep that in mind for memcmp().

    We can also send all this work off to the thread pools easily now with libdex/futures.

    Some work still remains if we want to use CTags for symbol resolution but I highly doubt we do.

    Anyway, having CTags is really more just about having an easy test case for the completion engine than “people will actually use this”.

  • Also write a new CTags miner which can build CTags files using whatever ctags engine is installed (universal-ctags, etc). The goal here is, again, to test the infrastructure in a super easy way rather than have people actually use this.

  • A new FoundrySymbolProvider and FoundrySymbol API which allows for some nice ergonomics when bridging to tooling like LSPs.

    It also makes it a lot easier to implement features like pathbars since you can foundry_symbol_list_to_root() and get a future-populated GListModel of the symbol hierarchy. Attach that to a pathbar widget and you’re done.

Foundry-GTK

  • Make FoundrySourceView final so that we can be a lot more careful about life-cycle tracking of related documents, buffers, and addins.

  • Use FoundryTextDocumentAddin to implement spellchecking with libspelling as it vastly improves life-cycle tracking. We no longer rely on UB in GLib weak reference notifications to do cleanup in the right order.

  • Improve the completion bridge from FoundryCompletionProvider to GtkSourceCompletionProvider. Particularly start on after/comment fields. We still need to get before fields setup for return types.

    Still extremely annoyed at how LSP works in this regards. I mean really, my rage that LSP is what we have has no bounds. It’s terrible in almost every way imaginable.

Builder

  • Make my Builder rewrite use new FoundrySourceView

  • Rewrite search dialog to use FoundrySearchEngine so that we can use the much faster VCS-backed file-listing + fuzzy search.

GtkSourceView

  • Got a nice patch for porting space drawing to GskPath, merged it.

  • Make Ctrl+n/Ctrl+p work in VIM emulation mode.

Sysprof

  • Add support for building introspection/docs. Don’t care about the introspection too much, because I doubt anyone would even use it. But it is nice to have documentation for potential contributors to look at how the APIs work from a higher level.

GUADEC

  • Couldn’t attend GUADEC this year, so wrote up a talk on Foundry to share with those that are interested in where things are going. Given the number of downloads of the PDF, decided that maybe sharing my weekly status round-up is useful.

  • Watched a number of videos streamed from GUADEC. While watching Felipe demo his new boxes work, I fired up the repository with foundry and things seem to work on aarch64 (Asahi Fedora here).

    That was the first time ever I’ve had an easy experience running a virtual machine on aarch64 Linux. Really pleasing!

foundry clone https://gitlab.gnome.org/felipeborges/boxes/
cd boxes/
foundry init
foundry run

LibMKS

  • While testing Boxes on aarch64 I noticed it is using the Cairo framebuffer fallback paintable. That would be fine except I’m running on 150% here and when I wrote that code we didn’t even have real fractional scaling in the Wayland protocol defined.

    That means there are stitch marks showing up for this non-accelerated path. We probably want to choose a tile-size based on the scale- factor and be done with it.

    The accelerated path shouldn’t have this problem since it uses one DMABUF paintable and sets the damage regions for the GSK renderer to do proper damage calculation.

The Foundry of Builder

I won’t be traveling this summer for GUADEC, so here is a quick rundown of what I would talk about if I were there.

Personally, I feel like Foundry has the potential to be far more useful than Builder alone. This is probably a good time to write about how it got here and where I intend to take it. Hopefully with your help!

A screenshot of the title page of the presentation, which is likely more accessible than this web page.
Read the Slides Here

Sysprof in your Mesa

Thanks to the work of Christian Gmeiner, support for annotating time regions using Sysprof marks has landed in Mesa.

That means you’ll be able to open captures with Sysprof and see the data along other useful information including callgraphs and flamegraphs.

I do think there is a lot more we can do around better visualizations in Sysprof. If that is something you’re interested in working on please stop by #gnome-hackers on Libera.chat or drop me an email and I can find things for you to work on.

See the merge request here.

Simplifying LSP Selection

With Foundry I want to make LSP management much easier than it currently is in Builder.

We have the foundry lsp run python3 command where python3 can be replaced with any language for which there is an installed LSP plugin. This will start an LSP using all the abstractions (including cross-container execution) and provide it via stdin/stdout.

But what happens when you have a half-dozen language servers for Python with new ones added every week? There is a simple builtin tool now.

Keep in mind the language identifiers should match GtkSourceView language identifiers.

# Make clangd the preferred LSP for C
foundry lsp prefer clangd c

# Make sourcekit-lsp preferred LSP for C++
foundry lsp prefer sourcekit-lsp cpp

# Make ruff the preferred LSP for Python3
foundry lsp prefer ruff python3

If there is a clear LSP that your project should be using by all contributors, add --project and it will update the value in the projects settings.

Filtering Containers in Ptyxis

Some people seem to have an outrageous number of containers on their system. That can create pretty bad performance with Ptyxis when it is using a GtkPopoverMenu to show you the container list.

Nightly will now use a custom popover backed by GtkListView which should help with that. Additionally, you can filter them now easily. To bring up the menu, alt+, can be used.

A screenshot of Ptyxis showing the popover menu for terminals including containers and profiles. You can filter them by typing into a search entry.