Status Week 48

This week was Thanksgiving holiday in the US and I spent the entire week quite sick with something that was not Covid nor Flu according to rapid tests.

Managed to get a few things done through sheer force of will. I don’t recommend it.

Red Hat

  • A lot of misdirection related to our international move towards France where my wife has family. The policies at Red Hat may mean that we need to come up with a strategy to keep a million-plus lines of code maintained if my only option to protect my family is to go the route of long-stay-visa (e.g. work not permitted).

    Some very lovely people have reached out and thank you for that!

    Touch base if you have options that would allow me to work from France if you like my approach to engineering. While I’d love to continue working on GNOME/GTK and related technologies, some things are more important and so I’m flexible.

    Lets all hope that the Corporate Leadership Team at Red Hat determines that supporting multi-ethnic families get safely out of the United States near their support network is of value to our mutual success.

Ptyxis

  • Triage a handful of issues. This project has a pretty high-velocity file-rate while having an pretty low “bug exists here” rate. This legitimately burns a lot of my time every week that could be spent creating things.

    So if you are going to file issues in projects like Ptyxis or Text Editor, please take the time to do basic binary-search on where the problem really exists. Points given just for trying.

    In fact, this is a rant about how you should do that in all aspects of your life. Cut the problems you have in half — and half again.

Libdex

  • Fumbled my way through adding a gi override to integrate DexFuture with asyncio. This allows for await some_future() but you need to make sure you have a GLib main context running as well and integrated with asyncio.

    Hopefully this means that you will soon be able to easily use all the nice Foundry APIs from Python with relative ease.

Foundry

  • Add support for communicating with SSH agent for simple signing requests. Was surprised to not find this in libssh2 but maybe I missed something. We already link against that for libgit2 support.

  • Add support for signing commits in the FoundryGitCommitBuilder commit helper. This supports GPG and SSH commit signing (no X509) because I have no need for it. Though the SSH form is tested best.

  • Iteration on tracking changes to untracked/unstaged/staged files while using the commit builder API.

  • Abstract commit signing into generic buffer signing so that it can be used for more things in the future (like signing tags).

  • Add support for staging/unstaging files, hunks, and lines.

  • New test-git-commit-builder-gtk to be able to test out the infrastructure for commit creation. It’s extremely scrappy but gets the job done enough to test the API out even if not very ergonomic.

  • New foundry_git_vcs_stash() helper to be like git stash.

  • Lots of new APIs around generating diffs, deltas, and patches.

  • Lots of refactoring of Git subsystem to make the combination of threading and convenient APIs easier to maintain.

Builder

  • Some work on git change management panel, diff viewer, etc

  • VCS history panel which is useful as auxiliary for files as well as for viewing diffs.

  • Details auxiliary panel for forge issues/merge-requests

  • Experiment with using AdwPreferencesGroup and friends for creating AdwBottomSheet style menus in narrow mode (e.g. mobile).

Libdex Futures from asyncio

One of my original hopes for Libdex was to help us structure complex asynchronous code in the GNOME platform. If my work creating Foundry and Sysprof are any indicator, it has done leaps and bounds for my productivity and quality in that regard.

Always in the back of my mind I hoped we could make those futures integrate with language runtimes.

This morning I finally got around to learning enough of Python’s asyncio module to write the extremely minimal glue code. Here is a commit that implements integration as a PyGObject introspection override. It will automatically load when you from gi.repository import Dex.

This only integrates with the asyncio side of things so your application is still responsible for integrating asyncio and GMainContext or else nothing will be pumping the DexScheduler. But that is application toolkit specific so I must leave that to you.

I would absolutely love it if someone could work on the same level of integration for GJS as I have even less experience with that platform.

Status Week 47

Ptyxis

  • Issue filed about highlight colors. Seems like a fine addition but again, I can’t write features for everyone so it will need a MR.

  • Walk a user through changing shortcuts to get the behavior they want w/ ctrl+shift+page up/down. Find a bug along the way.

  • Copy GNOME Terminal behavior for placement of open/copy links from the context menu.

  • Add a ptyxis_tab_grab_focus() helper and use that everywhere instead of gtk_widget_grab_focus() on the tab (which then defers to the VteTerminal). This may improve some state tracking of keyboard shift state, which while odd, is a fine enough workaround.

  • Issue about adding waypipe to the list of “network” command detection. Have to nack for now just because it is so useful in situations without SSH.

  • Issue about growing size of terminal window (well shrinking back) after changing foreground. Triaged enough to know this is still a GDK Wayland issue with too-aggressively caching toplevel sizes and then re-applying them even after first applying a different size. Punted to GTK for now since I don’t have the cycles for it.

Foundry

  • Make progress icon look more like AdwSpinner so we can transition between the two paintables for progress based on if we get any sort of real fractional value from the operation.

  • Paintable support for FoundryTreeExpander which is getting used all over Foundry/Builder at this point.

  • Now that we have working progress from LSPs we need to display it somewhere. I copied the style of Nautilus progress operations into a new FoundryOperationBay which sits beneath the sidebar content. It’s not a perfect replicate but it is close enough for now to move on to other things.

    Sadly, we don’t have a way with the Language Server Protocol to tie together what operation caused a progress object to be created so cancellation of progress is rather meaningless there.

  • Bring over the whole “snapshot rewriting” we do in Ptyxis into the FoundryTerminal subclass of VTE to make it easy to use in the various applications getting built on Foundry.

  • Fix FoundryJsonrpcDriver to better handle incoming method calls. Specifically those lacking a params field. Also update things to fix reading from LF/Nil style streams where the underlying helper failed to skip bytes after read_upto().

  • It can’t be used for much but there is a foundry mcp server now that will expose available tools. As part of this make a new Resources type that allows providing context in an automated fashion. They can be listed, read, track changes to list, subscribe to changes in a resource, etc.

    There are helpers for JSON style resources.

    For example, this makes the build diagnostics available at a diagnostics:// URI.

  • Make PluginFlatpakSdk explicitly skip using flatpak build before the flatpak build-init command in the pipeline should have run.

  • Allow locating PluginDevhelpBook by URI so that clicking on the header link in documentation can properly update path navigators.

  • Make FoundryPtyDiagnostics auto-register themselves with the FoundryDiagnosticManager so that things like foundry_diagnostic_manager_list_all() will include extracted warnings that came from the build pipeline automatically rather than just files with diagnostic providers attached.

  • Support for “linked pipelines” which allows you to insert a stage into your build pipeline which then executes another build pipeline from another project.

    For example, this can allow you to run a build pipeline for GTK or GtkSourceView in the early phase of your projects build. The goal here is to simplify the effort many of us go through working on multiple projects at the same time.

    Currently there are limitations here in that the pipeline you want to link needs to be setup correctly to land in the same place as your application. For jhbuild or any sort of --prefix= install prefix this is pretty simple an works.

    For the flatpak case we need more work so that we can install into a separate DESTDIR which is the staging directory of the app we’re building. I prefer this over setting up incremental builds just for this project manually as we have no configuration information to key off.

  • Add new FoundryVcsDiffHunk and FoundryVcsDiffLine objects and API necessary access them via FoundryVcsDelta. Implement the git plugin version of these too.

  • Add a new FoundryGitCommitBuilder high-level helper class to make writing commit tooling easier. Provides access to the delta, hunk, and lines as well as commit details. Still needs more work though to finish off the process of staging/unstaging individual lines and try to make the state easy for the UI side.

    Add a print-project-diff test program to test that this stuff works reasonably well.

Builder

  • Lots of little things getting brought over for the new editor implementation. Go to line, position tracking, etc.

  • Searchable diagnostics page based on build output. Also fix up how we track PTY diagnostics using the DiagnosticManager now instead of only what we could see from the PTY diagnostics.

Manuals

  • Merge support for back/forward mouse buttons from @pjungkamp

  • Merge support for updating pathbar elements when navigating via the WebKit back/forward list, also from @pjungkamp.

Systemd

  • Libdex inspired fiber support for systemd which follows a core design principle which is that fibers are just a future like any other.

    https://github.com/systemd/systemd/pull/39771

CentOS

  • Merge vte291/libadwaita updates for 10.2

GTK

  • Lots of digging through GdkWayland/GtkWindow to see what we can do to improve the situation around resizing windows. There is some bit of state getting saved that is breaking our intent to have a window recalculate it’s preferred size.

GNOME Settings Daemon

  • Took a look over a few things we can do to reduce periodic IO requests in the housekeeping plugin.

    Submitted a few merge requests around this.

GLib

  • Submitted a merge request adding some more file-systems to those that are considered “system file system types”. We were missing a few that resulted in extraneous checking in gsd-housekeeping.

    While researching this, using a rather large /proc/mounts I have on hand, and modifying my GLib to let me parse it rather than the system mounts, I verified that we can spend double-digit milliseconds parsing this file. Pretty bad if you have a system where the mounts can change regularly and thus all users re-parse them.

    Looking at a Sysprof recording I made, we spend a huge amount of time in things like strcmp(), g_str_hash(), and in hashtable operations like g_hash_table_resize().

    This is ripe for a better data-structure instead of strcmp().

    I first tried to use gperf to generate perfect hashes for the things we care about and that was a 10% savings. But sometimes low-tech is even better.

    In the end I went with bsearch() which is within spitting distance of the faster solutions I came up with but a much more minimal change to the existing code-base at the simple cost of keeping lists sorted.

    There is likely still more that can be done on this with diminishing returns. Honestly, I was surprised a single change would be even this much.

Red Hat

This week also had me spending a significant amount of time on Red Hat related things.

Status Week 46

Ptyxis

  • More back and forth on people who have issues with diacritics and ibus interaction. It’s extremely frustrating at times because the two places where stuff like this gets reported first is the text editor and terminal, when rarely those two applications have anything to do with the issue.

    In this case we might have a workaround by being a bit more explicit about focus grabs.

  • Merge support for changing profile on existing tab.

VTE

  • Back-and-forth on updating MR for PTY errno, merge to master, vte-0-82, possibly backport further for RHEL/CentOS.

  • Research some us dead key issues with diacritics and see if we can find where in VTE a problem could be. Text Editor apparently doesn’t replicate the same issue, so it’s possible we should fix something in VTE directly or in GTK IM abstractions. As mentioned in Ptyxis we can probably work around this for now.

Foundry

  • Now that foundry has support for API keys we need to have a mechanism to rotate those keys (and query for expiration). FoundryKeyRotator provides this abstraction to FoundrySecretService.

    This comes with foundry secret rotate HOST SERVICE which makes it easy to keep things up-to-date. It would be nice to do this automatically at some point though it’s rather annoying because you’ll get an email about it, at least from gitlab.

    To check the expiration, foundry secret check-expires-at is provided check again, takes a HOST SERVICE pair.

    Defaults to what the server wants for minimum key lifetime, or you can provide --expire-at=YYYY-MM-DD to specify expiration.

  • Implement symbol APIs for FoundryTextDocument which will power things like the symbol tree, what symbol is underneath my cursor, symbol path bars, and the like. Also added some command line tools for this so that it is easy to test the infrastructure when issues are inevitably filed.

    foundry symbol-tree FILE and foundry find-symbol-at FILE LN COL will quickly test the machinery for filing bug reports.

  • Updated the CTags parser (which is our usually “first implementation” symbol provider) out of simplicity. Allow it to generate data to GBytes instead of files on disk for usage on modified buffers. Also allow it to load an index from memory w/o touching disk for the other side of this index structure.

  • GCC changed the SARIF environment variable to EXPERIMENTAL_SARIF_SOCKET so track that for GCC 16 release.

  • Handy foundry_read_all_bytes(int fd) to exhaust a FD into a GBytes using the libdex AIO subsystem (io_uring, etc).

  • Prototype a tree-sitter based plugin for symbol providers so we can have some faster extractors at least for some very common languages.

  • Add FoundrySymbolLocator for locating symbols by a number of strategies that can be different based on the provider. Combine this with a new FoundrySymbolIntent to integrate into the intent system.

  • Implement FoundryPathNavigator for use in future pathbar work. Add subclasses for FoundrySymbolNavigator and FoundryDocumentationNavigator to make pathbars of those common hierarchies super easy from applications. A FoundryFileNavigator to be able to navigate the file-system. This ties in well with the intent system so activating path elements routes through intents like open-file intent, symbol intent, etc.

  • Implement FoundryPathBar and associated infrastructure for it to be abstracted in libfoundry-adw.

  • Implement LSP progress operations ($/progress and creation operations) so we can bridge them to FoundryOperation. Had to implement some missing bits of FoundryJsonrpcDriver while at it.

  • Improve support for LSP symbol APIs, particularly around support for hierarchical symbol trees. Newer revisions allow for symbols to contain other symbols rather than trying to recreate it from the containerName.

  • Discovered that there is an upper-limit to the number of GWeakRef that can be created and that number is surprisingly lower than I would expect. Sure there is extra overhead with weak refs, but having limits so low is surely a mistake.

  • Lots of work on LSP implementation to bridge things like diagnostics and symbols. It is amazing now much easier it is to do this time around now that I have fibers instead of callback noodles.

  • We have a much saner way of implementing buffer tracking this time around (especially after pushing commit notify into GTK) so the LSP integration around buffer synchronization has a cleaner implementation now.

  • Add tooltips support to the diagnostics gutter renderer.

  • A new “retained” listmodel which allows you to hold/release items in a list model and they’ll stay pinned until the hold count reaches zero. This is helpful for situations where you don’t want to affect an item while there is a mouse over something, or a popover is shown, that sort of deal. I didn’t opt for the scalable RBTree implementation yet, but someone could certainly improve it to do so.

Buider

  • Work on the new auxiliary panel design which will work a bit like it does in Text Editor but allow for panel groupings.

  • Symbols panel implementation work using the new symbol providers. Implement symbol intent handling as well to tie it all together.

  • Implement pathbar integration into text editor and documentation browser using new navigator integration.

  • Diagnostics panel for monitoring diagnostics across the active page without having to resort to scanning around either the global diagnostics or within the gutter.

  • Add annotation provider so we can show diagnostics/git-blame like we do now but in a much more optimized manner. Having diagnostics inline is new though.

  • Lots of styling work for the auxiliary panel to try to make it work well in the presence of a grid of documents. A bit harder to get right than in Text Editor.

  • Ergonomics for the messages panel (clipboard support, clearing history, etc).

  • Work on operation bay for long running operations similar to what Nautilus does. This will handle things like progress from LSPs, deployment to remote devices, etc.

CentOS

  • libadwaita backports. This one is rather frustrating because I’ve been told we can’t do sassc on the build infrastructure and since 1.6.3 libadwaita no longer generates the CSS on CI to be bundled with the tarball.

    So continue with the madness of about 60 patches layered on top of 1.6.2 to keep things progressing there. One patch won’t get in because of the CSS change which is unfortunate as it is somewhat a11y related.

    At the moment the options are (given the uncompromising no-sassc restriction), to keep back-porting and not get CSS changes, to pull in newer tarballs and generate the CSS on my machine and patch that in, or just keep doing this until we can *gestures* do something more compromising on the CentOS build infrastructure.

  • VTE backports for 0.78.6

GtkSourceView

  • Branched for 5.20 development so we can start adding new features.

  • Fix a gir annotation on GtkSourceAnnotation that had wrong transfer.

  • Make GtkSourceAnnotation right justified when it fits in the available space.

  • Add some nice styling to annotations so they are bit more pleasing to look at.

Status Week 45

Ptyxis

  • Handle some incoming issue reports which basically amounts to copying their question into google, searching, and copying the first result back. A reminder that we really need dedicated support channels that are not the issue tracker.

    But more importantly, how you move people there is still problematic. I could of course just tell them to go over “there”, but when the questions are so simple you end up taking the gentler approach and just answering it begrudgingly rather than coming off abrupt.

  • Issue reported about an async spin loop from GMainLoop after closing a window which has a really long paste actively feeding it. Looks like it needs to be addressed in VTE (more on that below), but also decided to raise priority of our fallback SIGKILL handler if SIGHUP failed.

  • Code review for incoming community feature to swap profiles on an active tab.

VTE

  • MR sent upstream to hopefully address the corner case of disposing a terminal widget while paste is ongoing.

  • Noticed that ibus-daemon is taking considerable CPU when running the ucs-decode test tool on Ptyxis/VTE. Probably something related to tracking input cursor positions w/ the text protocol or similar. Reached out to @garncho for any tricks we might be able to pull off from the VTE side of things.

Libdex

  • Looking into some strange behaviors with dex_thread_wait_for() when implementing the cross-thread semantics for libgit2 wrapping. As a result I changed where we control life-cycle for the waited upon future so that it is guaranteed longer than the DexWaiter.

  • Spend some time thinking about how we might approach a more generalized wrapper for existing GIO-like async functions. I was never satisfied with DexAsyncPair and perhaps now that we will gain a gdbus-codegen for Libdex we could gain a *.gir future wrapping codegen too.

  • Add a new DexFutureListModel which is a GListModel which is populated by a DexFuture resolving to a GListModel. Very handy when you have a future and want a GListModel immediately.

Foundry

  • Add FoundryFileRow so we have easy mechanics for the whole typing a file/directory path and browsing to it. Re-use the existing path collapse/expand stuff in Foundry to allow for niceties like ~/Directory. Fix a bunch of obnoxiousness that was in the Builder implementation of this previously.

  • Add foundry_operation_is_cancelled() to simplify integrating with external blocking/threaded code such as libgit2. This allows breaking out of a clone operation by checking for cancellation in the progress callbacks from the server but works for any sort of blocking API that itself has callback functions in its state machine.

  • Add new CLI formatter for GFlags to use value_nick instead of the whole enumeration value in UP_CASE.

  • Add FoundryBuildPipeline:build-system with discover from the FoundryBuildAddin before full addin loading occurs. This replicates the IdeBuildSystemDiscovery from Builder without having to use a secondary addin object.

  • Add priorities to FoundryBuildAddin so that we can pre-sort by priority before build-system discovery. That fixes loading the WebKit project out of the box so that CMakeLists.txt will be matched before Makefile if you configured in tree.

  • Implement write-back for FoundrySdkManager:sdk to the active configuration after checking of the config supports the SDK.

  • Implement write-back for buildconfig files.

  • Implement mechanics for write-back PluginFlatpakConfig. Also start on implementation for all the FlatpakSerializable types. This is made much more complicated by needing to keep track of the different subtle ways lists can be deserialized as well as referenced files which can be included. It will mean write-back may have a bucket of files to update. Though right now we only will do this for a few select fields so we can probably punt a bit.

  • Meson cleanup to all our tools that test various aspects of Foundry outside the confines of an entire IDE.

  • Non-destructive editing of flatpak manifests works for the core feature-set we need. It requires storing a lot of state while parsing manifests (and recursively their includes) but it seems to work well enough.

  • Setup needs-attention tracking for panels and pages.

  • Write a new foundry.1 manpage for inclusion in distributions that really want that provided.

  • Add :author property to FoundryForgeIssue and FoundryForgeMergeRequest so we can start presenting peer information in the Builder UI. Implement this for gitlab plugin.

  • Improve handling to changes in implicit-trailing-newline from file settings getting loaded/applied. That way we don’t run into weird behavior with file-change monitoring with ligbit2 getting something different than we’d expect.

  • Support for keyword search when listing issues/merge-requests through the forge interfaces.

  • Add FoundryDocumentationIntent for intent to read documentation.

  • Very basic support for Justfile using just to build. Currently it is sort of a cop-out because it uses build and clean instead of trying to track down [default] and what not. Contributions welcome.

Builder

  • Iteration on the new clone dialog based on Foundry including wiring up all the PTY usage, page navigation, etc.

  • Update builder-dark GtkSourceView style to fit in better with updated Adwaita palette. Even though it’s not using Tango palette for all the colors, we still want it to fit in.

  • Iteration on forge listings

  • Wire up needs attention for panels/pages

  • Setup forge splitbutton so we can jump to gitlab (or other forge) quickly or alternatively list issues/etc in app as we gain more forge support.

  • Implement manuals panel and page on top of the Foundry service FoundryDocumentationManager.

  • Implement browser page and port over old urlbar implementation. Still slogging through all the intricacies of navigation policy which I didn’t handle so well in Builder originally.

  • Work on bringing over path bar from Manuals so we can use it more generically for things like symbol paths and what not.

  • A lot of little things here and there that just need plumbing now that we’re in a world of futures and listmodels.

  • Lots of work on the updated greeter using Foundry. It looks mostly the same, just less madness in implementation.

Flathub

  • Investigate why older Builder is what is shown as published.

  • Merge PR to update Ptyxis to 49.2

  • Update Manuals to 49.1 since I missed the .0. Had a lot of libfoundry things to bring over as well. Needs an exceptions update to flathub linter.

Text Editor

  • Update builder style scheme

Libpanel

  • Update needs-attention style for the panel switcher buttons

Manuals

  • Make ctrl+k select all the existing text in the search entry as part of the focus-search action.

Status Week 44

Red Hat

In October I celebrated 10 years at Red Hat. I don’t think I ever worked somewhere for more than a few years before that. Considering I started working professionally in tech after I barely escaped high school, that’s nearly half of my career.

It is also a reminder to reflect on the people who helped me get there. All the people on IRC that helped me learn programming in the early days of #gtk and #gnome-hackers IRC channels. And Miguel specifically for all the random demo code he would send my way to read how something could be done.

I know I’m one of the lucky ones that was able to do this without a formal education. I don’t necessarily recommend that but it has its benefits too.

Ptyxis

  • Look into what app-id is used on Ubuntu to help users who can’t figure out where to put custom palettes. To make this a bit easier make sure we always have APP_ID in the troubleshooting data. Turned out to be missing [Light] section in their custom .palette.

    I can’t help but lament that this would be less of an issue if the org.gnome.* namespace where more allowed for people who are Foundation members. I’m both informed on why it is like this (a single situation that occurred by a former community member) and also disagree with the direction that was taken as a result of that.

  • Look into diacritics issue for input. Usually this ends up being a system configuration issue. There is an oddity about this one in that changing focus and coming back fixes it. Also using ibus directly instead of the wayland text protocol fixes it. Neither of these are ideal so it may have something to do with state tracking either in the GTK Wayland backend or in Mutter.

    Thankfully the great Carlos Garnacho is helping us track it down.

  • Look into why Silverblue has some users not seeing their containers which appear to be toolbox/podman related.

    Installed F43 Silverblue beta on my old testing laptop. Turns out I had an overzealous filter for the containers list. If you had more than one you’d be fine. This only affected 49 since that is the first release that gained support for the search filter box in the popover that otherwise looks like a GtkPopoverMenu.

    Made a release to get the fix out to F43 users quickly.

    Quickly I want to mention a feature that keyboard navigators will love. If you type Alt+comma the containers dialog will be displayed. If you hit Return you’ll land in the default host system (aka “My Computer”). If you type a few characters and hit Return you’ll land on the first container/profile that matches your keyword search.

Foundry

  • New API to simplify navigating directory listings since you may want “..” in certain contexts (like Builder’s directory page).

  • Add a FoundryFileSearchReplacement API to coordinate with the FoundryFileSearchMatch API. This makes it easy to do replacements based on search results in a manner like we do in GNOME Builder where you can tick on/off + replacement string w/ back-references. Of course to test this infrastructure I added a --replace= option to foundry grep so I can refactor quickly from Ptyxis.

  • Write documentation for most of libfoundry exposed classes.

  • Fix string array output for --format=json and improve it for regular text output.

  • Fix grep plugin to add multiple matches on a single line as separate matches in the result set. This fixes search/replace to handle all matches in a file.

  • New Terminal Intent for opening terminals within the application such as Builder. Add a TerminalService for exposing actions to the intent system to applications easily.

  • Setup using intents for search results because it will allow us to abstract things like “activate action”, “open file”, “spawn terminal”, “browse to symbol”, “read documentation”, etc.

  • Iteration on panel bar to try to give us a bit better interaction with the panel machinery in libfoundry-adw applications.

  • Improve fallback search when there is not a VCS available to accelerate index building. (We use git file listing which is almost instantaneous to build our initial fuzzy search index instead of trying to iterate the file tree).

  • Start on documentation bridge so doc search can show up in the regular search dialog.

  • Host SDK gained support for running processes through a systemd-run with scope/unit like Ptyxis does for new tabs (when systemd-run is discovered on the host). This just helps identify things easier.

  • Implement the whole PanelWidget::presented() machinery from libpanel into libfoundry-adw so that we can have Workspaces but also still be able to delay certain panel/page work until the user will actually see the widget. For example, the test suite panel needs to advance the build pipeline far enough to get introspection data which you don’t want to do unnecessarily.

  • Add a new way to track current phase of a build pipeline. It is now dynamically calculated based on the maximum of the stages which have :completed set to true (so log as others in the same phase have the same value).

  • Add new :eol property to FoundryDocumentationBundle so we can pass that information from SDKs into Manuals

  • Build system updates so that we still have more control over feature flags for compiling in/out what you need in Foundry. This is quite a burden for sure, but it allows more flexibility in where Foundry can be used. For example, Manuals doesn’t need the text subsystem or Git for example.

Builder

  • Trying a new design for various types of listings in the revamp. Directory listings, diagnostics listings, log panel, file search results, unit test panel, test output, etc.

  • Copied how I did shortcuts in Ptyxis for the revamp because it works a lot nicer from a configuration standpoint and visibility. It also makes it much easier to bind from .ui files into controllers as well as update GMenu with appropriate shortcuts.

  • Use g_file_trash() instead of delete when removing configurations.

GLib

  • Pull in a year old patch from Ulrich Drepper into a MR that can be merged to make g_filename_from_uri() handle some valid hostnames according to RFC 1123.

  • Extend the above patch to also support . later in labels so some questionable hostname like dev-ubu-25.10 in Ptyxis work.

Manuals

  • Show EOL information in documentation bundle installation dialog.

  • Allow showing/hiding EOL documentation bundles.

Releases

  • libpanel

  • Foundry

  • Builder

  • D-Spy

  • Manuals

  • Gom

Status Week 43

Got a bit side-tracked with life stuff but lets try to get back to it this week.

Libdex

  • D-Bus signal abstraction iteration from swick

  • Merge documentation improvements for libdex

  • I got an email from the Bazaar author about a crash they’re seeing when loading textures into the GPU for this app-store.

    Almost every crash I’ve seen from libdex has been from forgetting to transfer ownership. I tried hard to make things ergonomic but sometimes it happens.

    I didn’t have any cycles to really donate so I just downloaded the project and told my local agent to scan the project and look for improper ownership transfer of DexFuture.

    It found a few candidates which I looked at in detail over about five minutes. Passed it along to the upstream developer and that-was-that. Their fixu-ps seem to resolve the issue. Considering how bad agents are at using the proper ownership transfer its interesting it can also be used to discover them.

  • Add some more tests to the testsuite for future just to give myself some more certainty over incoming issue reports.

Foundry

  • Added a new Gir parser as FoundryGir so that we can have access to the reflected, but not compiled or loaded into memory, version of Gir files. This will mean that we could have completion providers or documentation sub-system able to provide documentation for the code-base even if the documentation is not being generated.

    It would also mean that we can perhaps get access to the markdown specific documentation w/o HTML so that it may be loaded into the completion accessory window using Pango markup instead of a WebKitWebView shoved into a GtkPopover.

    Not terribly different from what Builder used to do in the Python plugin for code completion of GObject Introspection.

  • Expanding on the Gir parser to locate gir files in the build, system, and installation prefix for the project. That allows trying to discover the documentation for a keyword (type, identifier, etc), which we can generate as something markdowny. My prime motivation here is to have Shift+K working in Vim for real documentation without having to jump to a browser, but it obviously is beneficial in other areas like completion systems. This is starting to work but needs more template improvements.

  • Make FoundryForgeListing be able to simplify the process of pulling pages from the remote service in order. Automatically populates a larger listmodel as individual pages are fetched.

  • Start on Gitlab forge implementation for querying issues.

    Quickly ran into an issue where gitlab.gnome.org is not servicing requests for the API due to Anubis. Bart thankfully updated things to allow our API requests with PRIVATE-TOKEN to pass through.

    Since validating API authorization tokens is one of the most optimized things in web APIs, this is probably of little concern to the blocking of AI scrapers.

  • Gitlab user, project, issues abstractions

  • Start loading GitlabProject after querying API system for the actual project-id from the primary git remote path part.

  • Support finding current project

  • Support listing issues for current project

  • Storage of API keys in a generic fashion using libsecret. Forges will take advantage of this to set a key for host/service pair.

  • Start on translate API for files in/out of SDKs/build environments. Flatpak and Podman still need implementations.

  • PluginGitlabListing can now pre-fetch pages when the last item has been queries from the list model. This will allow GtkListView to keep populating in the background while you scroll.

  • mdoc command helper to prototype discover of markdown docs for use in interesting places. Also prototyped some markdown->ansi conversion for nice man-like replacement in console.

  • Work on a new file search API for Foundry which matches a lot of what Builder will need for its search panel. Grep as a service basically with lots of GListModel and thread-pool trickery.

  • Add foundry grep which uses the foundry_file_manager_search() API for testing. Use it to try to improve the non-UTF-8 support you can run into when searching files/disks where that isn’t used.

  • Cleanup build system for plugins to make it obvious what is happening

  • Setup include/exclude globing for file searches (backed by grep)

  • Add abstraction for search providers in FoundryFileSearchProvider with the default fallback implementation being GNU grep. This will allow for future expansion into tooling like rg (ripgrep) which provides some nice performance and tooling like --json output. One of the more annoying parts of using grep is that it is so different per-platform. For example, we really want --null from GNU grep so that we get a \0 between the file path and the content as any other character could potentially fall within the possible filename and/or encoding of files on disk.

    Where as with ripgrep we can just get JSON and make each search result point to the parsed JsonNode and inflate properties from that as necessary.

  • Add a new IntentManager, IntentHandler, and Intent system so that we can allow applications to handle policy differently with a plugin model w/ priorities. This also allows for a single system to be able to dispatch differently when opening directories vs files to edit.

    This turned out quite nice and might be a candidate to have lower in the platform for writing applications.

  • Add a FoundrySourceBuffer comment/uncomment API to make this easily available to editors using Foundry. Maybe this belongs in GSV someday.

Ptyxis

  • Fix shortcuts window issue where it could potentially be shown again after being destroyed. Backport to 48/49.

  • Fix issue with background cursor blink causing transparency to break.

Builder

  • Various prototype work to allow further implementation of Foundry APIs for the on-foundry rewrite. New directory listing, forge issue list, intent handlers.

Other

  • Write up the situation with Libpeas and GObject Introspection for GNOME/Fedora.

Research

  • Started looking into various JMAP protocols. I’d really like to get myself off my current email configuration but it’s been around for decades and that’s a hard transition.

Libpeas and Introspection

One of the unintended side-effects of writing applications using language bindings is that you inherit the dependencies of the binding.

This made things a bit complicated when GIRepository moved from gobject-introspection-1.0 to girepository-2.0 as we very much want language bindings to move to the new API.

Where this adds great difficulty on maintainers is in projects like Libpeas which provides plug-in capabilities for GTK application developers across multiple programming languages.

In practice this has allowed applications like Gedit, Rhythmbox, and GNOME Builder to be written in C but load plugins from languages such as Python, Lua, JavaScript, Rust, C, C++, Vala, or any other language capable of producing a .so/.dylib/.dll.

A very early version of Libpeas, years before I took over maintaining the library, had support for GObject Introspection baked in. This allowed really interesting (at the time) tooling to perform dynamic method call dispatching using a sort of proxy object implemented at runtime. Practically nobody is using this feature from what I can tell.

But maintaining ABI being what it is, the support for it has long been part of the libpeas-1.x ABI.

A couple years ago I finally released a fresh libpeas-2.x ABI which removed a lot of legacy API. With objects implementing GListModel and PleasPluginInfo becoming a GObject, the need for libpeas-gtk dwindled. It’s extremely easy for your application to do everything provided by the library. Additionally, I removed the unused GObject Introspection support which means that libpeas-2.x no longer needs to link against gobject-introspection-1.0 (nor girepository-2.0).

One area where those are still used is the testsuite. This can complicate testing because we want to make sure that APIs work across language bindings but if the language binding uses a specific version of GObject Introspection that does not align with what the libpeas project is using for tests, it will of course lead to runtime disasters.

Such is the case with some language bindings. The Lua LGI project is scarcely maintained right now and still uses gobject-introspection-1.0 instead of girepository-2.0. I submitted patches upstream to port it over, but without an official maintainer well versed in C and language bindings there isn’t anyone to review and say “yes merge this”.

There is a fork now that includes some of the patches I submitted upstream, but the namespace is different so it isn’t a 1:1 replacement.

The PyGObject project has moved to girepository-2.0 upstream and that caused some breakage with applications in Fedora 42 that were still using the legacy libpeas-1.x ABI. For that reason, I believe the older PyGObject was released With Fedora 42.

If you are using libpeas-2.x in your application, you have nothing to fear with any of the language runtimes integrated with libpeas. Since libpeas doesn’t link against either introspection libraries, it can’t hurt you. Just make sure your dependencies and language bindings are all in sync and you should be fine.

If you are using libpeas-1.x still (2.x was released a little over 2 years ago) then you are in a much worse shape. Language bindings are moving (or have moved) to girepository-2.0 while libpeas cannot realistically be ported and maintain ABI. Too much is exposed as part of the library itself.

It is imperative that if you want to keep your application working that you are either on libpeas-2.x or you’re bundling your application in such a way that you can guarantee your dependencies are all on the same version of GObject Introspection.

Halfway ABI

There exists a sort of “half-way-ABI” that someone could work on with enough motivation which is to break ABI as a sort of libpeas-1.38. It would move to girepository-2.0 and all the ABI side-effects that come with it. Since the introspection support in libpeas-1.x is rarely used there should be little side-effects other than recompiling against the new ABI (and the build system transitions that go along with that).

In my experience maintaining the largest application using libpeas (being Builder), that is really a lot more effort than porting your applications to libpeas-2.x.

Is my app effected?

So in short, here are a few questions to ask yourself to know if you’re affected by this.

  • Does my application only use embedded plug-ins or plug-ins from shared-modules such as *.so? If so, then you are all set!
  • Do I use libpeas-1.x? If no, then great!
  • Does my libpeas-1.x project use Python for plug-ins? If yes, port to libpeas-2.x (or alternatively work suggested halfway-ABI for libpeas).
  • Does my libpeas-1.x or libpeas-2.x project use Lua for plug-ins? If yes, make sure all your dependencies are using gobject-introspection-1.0 only. Any use of girepository-2.0 will end in doom.

Since JavaScript support with GJS/MozJS was added in libpeas-2.x, if you’re using JavaScript plug-ins you’re already good. GJS recently transitioned to girepository-2.0 already and continues to integrate well with libpeas. But do make sure your other dependencies have made the transition.

How this could have been avoided?

Without a time machine there are only three options besides what was done and they all create their own painful side-effects for the ecosystem.

  1. Never break ABI even if your library was a stop gap, never change dependencies, never let dependencies change dependencies, never fix anything.
  2. When pulling GObject Introspection into the GLib project, rename all symbols to a new namespace so that both libraries may co-exist in process at the same time. Symbol multi-versioning can’t fix overlapping type name registration in GType.
  3. Don’t fix any of the glaring issues or inconsistencies when pulling GObject Introspection into GLib. Make gobject-introspection-1.0 map to the same thing that girepository-2.0 does.

All of those have serious side-effects that are equal to if not worse than the status-quo.

Those that want to “do nothing” as maintainers of their applications can really just keep shipping them on Flatpak but with the Runtime pinned to their golden age of choice.

Moral of the story is that ABI’s are hard even when you’re good at it. Doubly so if your library does anything non-trivial.

mi2-glib

At Red Hat we are expected to set, and meet, goals each quarter if we want our “full” bonus. One of those is around introducing AI into our daily work. You’ve probably seen various Red Hat employees talking about using AI. It’s hard to deny there is financial incentive to do so.

Astute students of behavioral science know that humans work harder to not lose something than to gain something new. Arguably it’s only a “mandate to use AI” if you are entitled to the revenue so attaching it to your bonus is a convenient way to both “not be a mandate” and take advantage of the human behavior to not lose something.

La mujer del César no solo debe ser honesta, sino parecerlo.

Conveniently, I got pretty fed up with Debug Adapter Protocol (DAP for short) this week. So I wondered how hard it would be to just talk the MI2 protocol like Builder does (using the GPLv3+ gdbwire) but with a libdex-oriented library. It could potentially make integrating GDB into Foundry much easier.

Red Hat has so generously given us “ephemeral credits” at Cursor.com to play with so I fired up the cursor-agent command line tool and got started.

My goal here is just to make a minimal API that speaks the protocol and provides that as DexFuture with a very shallow object model. The result for that is mi2-glib. This post is mostly about how to go about getting positive results from the bias machine.

Writing a fully featured library in a couple hours is certainly impressive, regardless of how you feel about AI’s contributions to it.

Before I continue, I want to mention that this is not used in Foundry and the GDB support which landed today is using DAP. This was just an experiment to explore alternatives.

Create Project Scaffolding

To get the project started I used Foundry’s shared library template. It creates a C-based project with pkg-config, GObject Introspection support, Vala *.vapi generation, versioning with ABI mechanics, and gi-doc based documentation. Basically all the tricky things just work out of the box.

You could probably teach the agent to call this command but I choose to do it manually.

foundry template create library
Location[/home/christian]: .

The name for your project which should not contain spaces
Project Name: mi2-glib

The namespace for the library such as "Mylib"
Namespace: Mi2
 1: No License
 2: AGPL 3.0 or later
 3: Apache 2.0
 4: EUPL 1.2
 5: GPL 2.0 or later
 6: GPL 3.0 or later
 7: LGPL 2.1 or later
 8: LGPL 3.0 or later
 9: MIT
10: MPL 2.0
License[7]:
Version Control[yes]: 
mi2-glib/meson.options
mi2-glib/meson.build
mi2-glib/lib/meson.build
mi2-glib/testsuite/meson.build
mi2-glib/lib/mi2-glib.h
mi2-glib/lib/mi2-version.h.in
mi2-glib/lib/mi2-version-macros.h
mi2-glib/testsuite/test-mi2-glib.c
mi2-glib/doc/mi2-glib.toml.in
mi2-glib/doc/urlmap.js
mi2-glib/doc/overview.md
mi2-glib/doc/meson.build
mi2-glib/README.md
mi2-glib/LICENSE
mi2-glib/.foundry/.gitignore
mi2-glib/.foundry/project/settings.keyfile
mi2-glib/.git/objects
mi2-glib/.git/refs/heads
mi2-glib/.git/HEAD

Building Context

After that I added an AGENTS.md file that described how I want code to be written. I gave it some examples of my C style (basically opinionated GTK styling). That is things like preferring autoptr over manual memory management, how to build, how to test, etc. Consider this the “taste-making” phase.

The next thing I needed to do was to ensure that it has enough examples to be able to write libdex-based GObject code. So I copied over the markdown docs from libdex and some headers. Just enough for the agent to scan and find examples to prime how it should be used. Update your AGENTS.md to note where that documentation can be found and what it can expect to find there. That way the agent knows to look for more details.

Just as a matter of practicality, it would be great if we could have gi-doc generate markdown-formatted descriptions of APIs rather than just HTML.

Now that is all good and fun, but this is about mi2 now isn’t it? So we need to teach the agent about what mi2 is and how it works. Only then will it be able to write something useful for us.

GDB conveniently has a single-file version of the documentation so we need not piss off any network admins with recursive wget.

That left me with a big HTML document with lots of things that don’t matter. I wanted something more compact that will better fit into the context window of the agent. So I asked the agent to look over the HTML file and amalgamate a single gdb.md containing the documentation in a semi-structured format with markdown. That is of course still a lot of information.

Next, I went another level deeper where I asked it to extract from gdb.md information about the mi2 protocol. Any command in the mi2 protocol or related command in GDB is important and therefore should be included in a new mi2.md document. This will serve as the basis of our contextual knowledge.

Knowing the Problem Domain

Using an agent is no replacement for knowing the problem domain. Thankfully, I’ve written dozens of socket clients/servers using GIO primitives. So I can follow my normal technique.

First we create a client class which will manage our GIOStream. That may be a GSocketConnection or perhaps even a GSimpleIOStream containing memory streams or file streams to enable unit testing.

Then we move on to a GDataInputStream which can read the format. We ask the agent to subclass appropriately and provide a DexFuture based wrapper which can read the next message. It should have enough context at this point to know what needs to be read. Start simple, just reading the next message as a bag of bytes. We parse it into structured messages later.

After that do the same for a GDataOutputStream subclass. Once we have that we can ask the agent to wrap the input/output streams inside the client. All pretty easy stuff for it to get right.

Where I found it much easier to write things myself was dealing with the mechanics of a read-loop that will complete in-flight operations. Small price to pay to avoid debugging the slop machine.

After these mechanics are in place we can start on the message type. I knew from my previous debugger work in Builder that there would be a recursive result type and a toplevel container for them. I decided to name these Message and Result and then let the agent generate them. Even the first try had something “usable” even if I would say not very tasteful. It kept trying to do manual memory management. It left lots of “TODO” in place rather than actually doing the thing.

Instead of trying to describe to the agent how its implementation was ugly and incomplete I realized my time would be better spent letting it figure it out. So I shifted my focus towards a test suite.

Setting up a Test Suite

I saw the biggest improvement from the agent after I setup a test suite. This allowed the agent to run ninja test to check the results. That way it wasn’t just about satisfying GCC compiler warnings but also that it was functional.

I would start this as early as you can because it pays dividends. Make sure you provide both good and bad inputs to help guide it towards a better implementation.

The biggest win was creating tests for the input stream. Create different files with good and bad input. Since they wrap a base stream it is pretty easy to mock that up with a GFileInputStream or GMemoryOutputStream and verify the results. If the tests provide enough context through debug messages the agent can feed that back into why the error occurred. This made the iteration loop remarkably fast.

Another extremely valuable thing to do is to setup some type of sanitizer in your build configuration. I often build with meson configure -Db_sanitize=address to help catch memory errors. But also do things like assert object finalization in your testsuite to make sure that leaks don’t hide them.

Anyway, writing this post took about as long as designing the core parts of the library and that, I guess in some small way, is pretty neat.

Status Week 39

It’s the time of year where the Oregon allergens has me laid out. Managed to get some stuff done while cranking up the air purifier.

VTE

  • Work on GtkAccessibleHypertext implementation. Somewhat complicated to track persistent accessible objects for the hyperlinks but it seems like I have a direction to move forward.

Ptyxis

  • Fix CSS causing too much padding in header bar 49.0

  • Make --working-directory work properly with --new-window when you want you also are restoring a session.

  • Add support for cell-width control in preferences

  • Move an issue to mutter for more triage. Super+key doesn’t get delivered to the app until it’s pressed a second time. Not sure if this is by design or not.

Foundry

  • Merge support for peel template in GTK/Adwaita

  • Lots of work on DAP support for FoundryDebugger

  • Some improvements to FOUNDRY_JSON_OBJECT_PARSE and related helper macros.

  • Handle failure to access session bus gracefully with flatpak build pipeline integration.

  • Helper to track changes to .git repository with file monitors.

  • Improve new SARIF support so we can get GCC diagnostics using their new GCC 16.0 feature set (still to land).

  • Lots of iteration on debugger API found by actually implementing the debugger support for GDB this time using DAP instead of MI2.

  • Blog post on how to write live-updating directory list models.

  • Merge FoundryAdw which will contain our IDE abstractions on top of libpanel to make it easy to create Builder like tooling.

  • Revive EggLine so I have a simple readline helper to implement a testing tool around the debugger API.

  • Rearrange some API on debuggers

  • Prototype a LLDB implementation to go along with GDB. Looks like it doesn’t implement thread events which is pretty annoying.

  • Fix issue with podman plugin over-zealously detecting it’s own work.

Builder

  • Make help show up in active workspace instead of custom window so that it can be moved around.

Libdex

  • Write some blog posts

  • Add an async g_unlink() wrapper

  • Add g_find_program_in_path() helper

Manuals

  • Fix app-id when using devel builds

  • Merge support for new-window GAction

Libpanel

  • Merge some improvements to the changes dialog

Text Editor

  • Numerous bugs which boil down to “ibus not installed”

D-Spy

  • Issue tracker chatter.