Accelerators, a history lesson

It’s good to dive into our shared history every now and again to learn something new. We want to build a customized shortcut engine for Builder and that means we need to have a solid understanding of all the ways to activate shortcuts in GTK+. So the following is a list of what I’ve found, as of GTK+ 3.22. I’ve included some pros and cons of each based on my experience using them in particularly large applications.

GtkAccelGroup and GtkAccelMap

The GtkAccelGroup class is remnant from the days before GtkUIManager was deprecated. It is a structure that is attached to a top-level GtkWindow and maps “paths” to closures. It’s purpose was to be used with a GtkAccelMap to ultimately bind accelerators (such as “<control>q”) to a closure (such as gtk_main_quit).

The GtkAccelMap is a singleton that maps accelerators to paths. When activating a keybinding, the accelerator machinery will try to match the path with one registered in the GtkAccelGroup on the window.

Pros

  • Widgets can be activated no matter what the focus widget. However, the window must be focused.
  • It is simple to display keyboard accelerators next to menu items.
  • Creating an accelerator editor is straight forward. Lookup accel path, map to accelerator.

Cons

  • Not deprecated, but much of the related machinery is deprecated.
  • Fairly complex API usage. It doesn’t seem to be designed to be used by application developers.

Signal Actions

Signal actions are GObject signals that contain the G_SIGNAL_ACTION bit set. By marking this bit, we allow the GTK+ machinery to map an accelerator to a particular signal on the GtkWidget (or any ancestor widget in the hierarchy if it matches).

In the GTK+ 2.x days, we would control these keybindings using a gtkrc file. But today, we use GTK+ CSS to create binding sets and attach them to widgets using selectors. This is how the VIM implementation in Builder is done.

It should be noted that signal actions (and binding them from gtkrc/css) were meant to provide a way to provide key themes to GTK+ such as an emacs mode. It was not meant to be the generic way that applications should use to activate keybindings.

Pros

  • You can use CSS selectors to apply binding sets (groups of accelerators) to widgets.
  • Accelerators can include parameters to signals, meaning you can be quite flexible in what gets activated. With enough bong hits you’ll eventually land at vim.css.

Cons

  • Only the highest-priority CSS selector can attach binding sets to the widget. So the highest-priority needs to know about all binding sets that are necessary. That reduces the effectiveness of this as a generic solution.
  • This relies on the focus chain for activation. That means you can’t have a signal activate if another widget focused such as a something in the header bar.
  • It feels like a layer violation. I could make arguments on both side of the fence for this, which is sort of indicative of the point.
  • We have to “unbind” various default accelerators on widgets that collide with our key themes. This means that “key themes” are not only additive, but destructive as well.

GAction

Years back, we wanted the ability to activate actions inside of a program in a structured way. This allowed for external menu items as well as simplifying the process of single-instance applications. This was integrated into the GtkWidget hierarchy as well. Using gtk_application_set_accels_for_action() you can map an accelerator to a GAction. Using the current widget focus, the GTK+ machinery will work its way up the widget hierarchy until it finds a matching action. For example, if a widget has a “foo” GActionGroup associated with it, and that group contains an action named “bar”, then mapping “<control>b” to “foo.bar” would activate that action.

Pros

  • Actions can have state, meaning that we can attach them to things like toggle buttons.
  • Application level actions can be activated no matter what window is focused.
  • It’s fairly easy to reason about what actions are discoverable and activatable from a node in the widget tree. Especially with GTK+ Inspector integration.
  • Theoretically we could rebuild a11y on top of GAction and fix a lot of synchronous IPC to be async similar to how GMenu integrates with external processes. This would require a lot of external collaboration on various stacks though, so it isn’t an immediate “Pro”.

Cons

  • Action activation requires the focus hierarchy to activate the action. This means to get something activatable from the entire window we need to attach the actions to the top-level. This can be a bit inconvenient, especially in multi-document scenarios which do not share widgetry.
  • Actions are rather verbose to implement. Doing them cleanly means implementing proper API to perform the action, and then wrapping those using GActionEntry or similar. Additionally, to propagate state, the base API needs to know about the action so it can update action state. Coming up with a strategy for who is the “owner” of state is challenging for newcomers.
  • The way to attach accelerators is via gtk_application_set_accels_for_action() which is a bit weird for widgets that would want to provide accelerators. It means the applications are given more freedom to control how accelerators work at the cost of having to setup a lot of their own mechanics for activation.

What do we need in Builder?

Builder certainly lies on the “more complex” side of things in terms of what is needed from the toolkit for accelerators. Let’s go over a few of those necessities to help create a wishlist for whatever code gets written.

VIM and Emacs Keybindings

These will never be 100% perfect, but we need reasonable implementations that allow Emacs and VIM users not scream at their computer while trying to switch to a new tool. That means we need to hit the 80% really well, even if that last 20% is diminishing returns. Today, we’ve found a way to mostly do this using G_SIGNAL_ACTION and a lot of custom GTK+ CSS. The con mentioned above of GTK+ CSS requiring that the -gtk-key-bindings: CSS property know about all binding sets that need to be applied from the selector make it unrealistic for us to keep pushing this further.

Custom Accelerators

We want the user to be able to start from a basic “key theme” such as Gedit, VIM, or Emacs and apply their own overrides. This means that our keybinding registration cannot be static, but flexible to account for changes while the application is running. We also need to know when there are collisions with other accelerators so that we can let the user know what they are doing may have side-effects.

GtkShortcutsWindow Integration

In Builder we do not account for the “key theme” in the shortcuts window. This can be confusing to users of Emacs and Vim mode as what they see may not actually activate the desired action. We would like a way to map the accelerators available (grouped and with proper sections) to be automatically reflected in our GtkShortcutsWindow.

Elevating Accelerator Visibility

The shortcuts window is nice, but making the accelerators available to the user while they are exploring the interface is far more beneficial (based on my experiences). So whatever we end up implementing needs to make it fairly trivial to display accelerators in the UI. Today, we do that manually by hard-coding these accelerators into the GtkBuilder UI files. That is less than ideal.

Window-wide Activation

If I’m focused on the source code editor, I may want to activate an accelerator from a panel that is visible. For example, <control><shift>F to activate the documentation search.

Using actions to do this today would require that all panel plugins register their actions on the top-level as well as keybindings using gtk_application_set_accels_for_action(). While this is acceptable for the underlying machinery, it’s almost certainly not the API we want to expose to plugin developers in Builder. It’s tedious and prone to breakage when multiple “key themes” are in play. So we need an abstraction layer here that can use the appropriate strategy for what the plugin has in mind and also allows for different accelerators based on the “key theme”.

Re-usability

There are other large applications in the GNOME eco-system that I care about. GIMP is one of those projects that has a hard time moving to GTK+ 3.x due to a few reasons, some of which are out of our control upstream. The sheer size of the application makes it difficult. Graphical tools often are full of custom widgetry which necessarily dives into implementation details. Another reason is how important accelerators are to an immersive, creativity-based application. I would like whatever we create to be useful to other projects (after a few API iterations). GIMP’s GTK+ 3 port is one obvious possibility.

Builder Rust

With Federico’s wonderful post on Rust’ifying librsvg I guess it makes sense to share what I’ve been doing the last couple of days.

I’ve been keeping my eye on Rust for quite a while. However, I’ve been so heads down with Builder the last two years that I haven’t really gotten to write any or help on integration into our platform. Rust appears to take a very pragmatic stance on integration with systems code (which is primarily C). The C calling convention is not going anywhere, so at some point, you will be integrating with some part of a system that is “C-like”. Allowing us to piecemeal upgrade the “Safety” of our systems is much smarter than rewrite-the-universe. This pragmatism is likely due to the realities of Rust’s birth at Mozilla. It’s a huge code-base, and incrementally modernizing it is the only reality that is approachable.

We too have a lot of code. And like many other projects, we care about being language agnostic to a large degree. In the early days we might have chosen C because it was the only toolchain that worked reliably on Linux (remember C++ pre-2000? or LinuxThreads?) but today we still care about “Language Interopability” and C is the undeniable common denominator.

One way in which we can allow the interopability that we desire and the Safety we need is to start approaching some of our problems like Federico has done. The C calling convention is “Safe”. That is not where zero-day bugs come from. They come from hastily written C code. It is perfectly fine to write our interfaces in C (which can be our basis for GObject Introspection) but implement Safety critical portions in Rust.

This is exactly what I’d like to see from the hundreds of thousands of lines of C I’ve written over the years. We need to identify and fix the Satefy Critical portions. GStreamer is already on the right track here by looking at codec and plugin implementations in Rust. I’m not sure how far they will go with adapting to Rust, but it will be one of the best case-studies we will have to learn from.

So because of this desire to look at building a Safer Platform for developers and users alike, I’ve decided to start adding Rust support to Builder. Thanks to the hard work of the Rust team, it’s a fairly easy project from our side. There is a new Language Server Protocol 2.0 that was worked on by various people at Microsoft, Red Hat, and elsewhere. The new rustls was announced last week and uses this protocol. So I implemented support for both as quick as I could, and now we have something to play with.

Because of the Language Server Protocol, our Rust plugin is tiny. It is essentially a glorified supervisor to handle subprocess crashes and some binding code to connect our stdin/stdout streams to the Language Server Protocol client in Builder. See for yourself.

There is a bunch more work for us in Builder to make it a great Rust IDE. But if people start playing with the language and are willing to work on Builder and GNOME to improve things as a community, we can build a Modern, Safe, and Elegant developer platform.

The big ticket next steps for those that want to contribute to Rust support would include:

  • Cargo build system support. I do believe that ebassi started on something here. But I need to circle back around and sync up.
  • Symbol Tree needs improvements.
  • Semantic highlighter (which we can implement using fuzzy symbols from the symbol tree until a real protocol comes along).
  • Upstream rustls needs work too to get us the features we want. So Rustaceans might want to spend some time helping out upstream.
  • We need to simplify the glue code from Rust←→GObject so that it is dead simple to wrap Rust code in a GObject-based library (where we get our free language interopability).
  • … and of course all the other Builder plumbing that needs to happen in general. See our list of projects that need work.
This image depicts control+clicking on a symbol to jump to its definition. +period or :gd in Vim mode also work.
This image depicts control+clicking on a symbol to jump to its definition. <alt>+period or :gd in Vim mode also work.
This image shows diagnostics displayed over source code.
This image shows diagnostics displayed over source code.
This image shows completion of fields from a struct.
This image shows completion of fields from a struct.
This image shows the Symbol Tree on the right containing elements from the Rust document.
This image shows the Symbol Tree on the right containing elements from the Rust document.

JCON – Part 2

I just finished the JCON_EXTRACT() support and it is already making my consumption of JSON easier. Here is an example:

g_autoptr(JsonNode) node = NULL;
JsonArray *ar = NULL;
gboolean success;

node = JCON_NEW (
  "foo", "{",
    "bar", "[", JCON_INT (1), JCON_INT (2), "]",
  "}"
);

success = JCON_EXTRACT (node,
  "foo", "{",
    "bar", JCONE_ARRAY (ar),
  "}"
);

And for now, you can just copy/paste the jcon.c and jcon.h files into your project, but I’d expect to come up with a patch we can push into json-glib at some point. It really belongs there.

JCON

Years ago during my tenure at MongoDB I worked on a couple libraries. Notably the libbson and mongo-c-driver libraries. One neat feature we had was this concept called BCON which stood for BSON C Object Notation. It was a succinct format for creating BSON documents that made it easier to reason about what you were creating.

So I decided to do the same thing this evening around json-glib because I found I was writing a lot of code to create objects/arrays/etc.

It looks something like this

g_autoptr(JsonNode) params = NULL;

params = JCON_NEW (                                                      
  "changes", "["                                                         
    "{",                                                                 
      "uri", JCON_STRING (src_uri),                                      
      "type", JCON_INT (FILE_CHANGE_TYPE_DELETED),                       
    "}",                                                                 
    "{",                                                                 
      "uri", JCON_STRING (dst_uri),                                      
      "type", JCON_INT (FILE_CHANGE_TYPE_CREATED),                       
    "}",                                                                 
  "]"                                                                    
);

Since this uses va_list it’s technically less type safe than your other options. But it uses some magic struct initializers to get things in a situation where we can bail at runtime if you did something wrong.

The other half that I’m currently still missing is the extraction support. If you replace JCON_NEW() with JCON_EXTRACT(node) and the values with pointers to values, you can quickly extract documents.

But that’s not done yet… so for next time…

jcon.h jcon.c

Sysprof Plans for 3.24

The 3.24 cycle is just getting started, and I have a few plans for Sysprof to give us a more polished profiling experience in Builder. The details can be found on the mailing list.

In particular, I’d love to land support for visualizers. I expect this to happen soon, since there is just a little bit more to work through to make that viable. This will enable us to get a more holistic view of performance and allow us to drill into callgraphs during a certain problematic period of the profile.

Once we have visualizer support, we can start doing really cool things like extracting GPU counters, gdk/clutter frame-clock timing, dbus/cpu/network monitors and whatever else you come up with.

A CPU visualizer displayed above the callgraph to provide additional context to the profiled execution.
A CPU visualizer displayed above the callgraph to provide additional context to the profiled execution.

Additionally we have some work to do around getting access to symbols when we are running in binary stripped environments. This means we can upload a stripped binary to your IoT/low-power device to profile, but have the instruction-pointer-to-symbol resolver happen on the developers workstation.

As I just alluded to, I’d love to see remote profiling happen too. There is some plumbing that needs to occur here, but in general it shouldn’t be terribly complicated.

Builder Nightly Flatpak

First off, I’ll be in Portland at the first ever LAS giving demos and starting on Builder features for 3.24.

For a while now, you’ve been able to get Builder from the gnome-apps-nightly Flatpak repository. Until now, it had a few things that made it difficult to use. We care a whole lot about making our tooling available via Flatpak because it is going to allow us to get new code into users hands quicker, safer, and more stable.

So over the last couple of weeks I’ve dug in and really started polishing things up. A few patches in Flatpak, a few patches in Builder, and a few patches in Sysprof start getting us towards something refreshing.

Python Jedi should be working now. So you can autocomplete in Python (including GObject Introspection) to your hearts delight.

Jhbuild from within the Flatpak works quite well now. So if you have a jhbuild environment on your host, you can use the Flatpak nightly and still target your jhbuild setup.

One of the tricks to being a module maintainer is getting other people to do your work. Thankfully the magnificent Patrick Griffis came to the rescue and got polkit building inside of Flatpak. Combined with some additional Sysprof patches, we have a profiler that can run from Flatpak.

Another pain point was that the terminal was inside of a pid/mount/network namespace different than that of the host. This meant that /usr/lib was actually from the Flatpak runtime, not your system /usr/lib. This has been fixed using one of the new developer features in Flatpak.

Flatpak now supports executing programs on the host (a sandbox breakout) for applications that are marked as developer tools. For those of you building your own Flatpaks, this requires --allow=devel when running the flatpak build-finish command. Of course, one could expect UI/UX flows to make this known to the user so that it doesn’t get abused for nefarious purposes.

Now that we have access to execute a command on the host using the HostCommand method of the org.freedesktop.Flatpak.Development interface, we can piggy back to execute our shell.

The typical pty dance, performed in our program.

/* error handling excluded */
int master_fd = vte_pty_get_fd (pty);
grantpt (master_fd);
unlockpt (master_fd);
char *name = ptsname (master_fd);
int pty_fd = open (name, O_RDWR, 0);

Then when executing the HostCommand method we simply pass pty_fd as a mapping to stdin (0), stdout (1), and stderr (2).

On the Flatpak side, it will check if any of these file-descriptors are a tty (with the convenient isatty() function. If so, it performs the necessary ioctl() to make our spawned process the controller of the pty. Lovely!

So now we have a terminal, whose process is running on the host, using a pty from inside our Flatpak.

Quick Highlight

Martin Blanchard put together a new “quick highlight” plugin for Builder this last week. It was a great example of how to submit a new feature, so I just wanted to highlight it here. Post to bugzilla, attach a patch, and we will review quickly and help with any additional integration that might be necessary.

Thanks Martin!

Highlights words matching the current selection
Highlights words matching the current selection

Sysprof + Builder

After the GNOME 3.20 cycle completed I started revamping Sysprof. More here, here, and here. The development went so smoothly that I did a 3.20 release a couple of weeks later.

A primary motivation of that work was rebuilding Sysprof into a set of libraries for building new tools. In particular, I wanted to integrate Sysprof with Builder as our profiler of choice.

On my flight back from GUADEC I laid the groundwork to integrate these two projects. As of Builder 3.21.90 (released yesterday) you can now profile your project quite easily. There are  more corner cases we need to handle but I consider those incremental bugs now.

Some of our upcoming work will be to integrate the Sysprof data collectors with Python and Gjs. The Gjs implementation is written, it just needs polish and integration with upstream. I think it will be fantastic once we have a compelling profiling story weather you are writing C, C++, Vala, Python, or Gjs.

We’ve also expanded the architectures supported by Sysprof. So I expect by time 3.22 is released, Sysprof will support POWER8, ARM, ARM64, mips, and various others as long as you have an up to date Linux kernel. That is an important part of our future plans to support remote profiling (possibly over USB or TCP). If you’re interested in working on this, contact me! The plumbing is there, we just need someone with time and ambition to lead the charge.

Selecting the Profiler

Builder's Callgraph View