Rendering Text with Glyphy

HarfBuzz 4.0 was recently released and it gained the ability to draw shapes using hb_draw(). Recently Behdad updated Glyphy to use that new HarfBuzz API which allows us to draw fonts on the GPU without involving FreeType.

I created a branch and along with Matthias we taught GTK’s OpenGL renderer to use Glyphy to store size-independent descriptions of glyphs in GL texture atlases which are drawn with shaders. All the magic really comes from Glyphy, so this was largely an integration project.

Contrast this with how things worked previously, which is to render the glyph for a particular size and x/y shift on the CPU and upload that to a texture atlas. The glyph is then drawn by using the coloring shader to apply color while copying into place. This has all sorts of drawbacks like pixel alignment restrictions and needing a copy of every glyph at every size and shift you need to render in the frame in a texture atlas (which can be reused across frames). Just as an example, if you wanted to animate the size of some text, you’d have a pretty expensive operation every frame and still run into perceived jitter as you align to pixel boundaries each frame.

With Glyphy, we have a single version of the glyph for any size in the texture atlas, stored as encoded arc lists generated from SDF information (as opposed to uploading SDF information to the texture as some other implementations have done). This results in incredibly crisp rendering at any scale or offset (where as other implementations using SDF information in textures tend to create artifacts).

After just a few days of tinkering with it, it seems good enough to actually render Text Editor.

There is certainly more work to be done before this type of advancement can be enabled by default, but it was a fun prototype to be sure.

Modernizing GTK’s macOS backend (again)

Since the early days of working on the macOS backend for GTK 4 I knew eventually we’d have to follow suit with what the major browsers were doing in terms of drawing efficiency. Using OpenGL was (while deprecated, certainly not going anywhere) fine from a performance standpoint of rendering. But it did have a few drawbacks.

In particular, OpenGL (and Metal afaik) layers don’t have ways to damage specific regions of the GPU rendering. That means as we’d flip between front and back buffers, the compositor will re-composite the whole window. That’s rather expensive for areas that didn’t change, even when using a “scissor rect”.

If you’re willing to go through the effort of using IOSurface, there does exist another possibility. So this past week I read up on the APIs for CALayer and IOSurface and began strapping things together. As a life-long Linux user, I must say I’m not very impressed with the macOS experience as a user or application author, but hey, it’s a thing, and I guess it matters.

The IOSurface is like a CPU-fronted cache on a GPU texture. You can move the buffer between CPU and GPU (which is helpful for software rendering with cairo) or leave it pretty much just in the GPU (unless it gets paged out). It also has a nice property that you can bind it to an OpenGL texture using GL_TEXTURE_RECTANGLE. Once you have a texture, you can back GL_FRAMEBUFFER with it for your rendering.

That alone isn’t quite enough, you also need to be able to attach that content to a layer in your NSWindow. We have a base layer which hosts a bunch of tiles (each their own layer) whose layer.contents property can be mapped directly to an IOSurfaceRef, easy peasy.

With that in place, all of the transparent regions use tiles limited to the transparent areas only (which will incur alpha blending by the compositor). The rest of the area is broken up into tiles which are opaque and therefore do not require blending by the compositor and can be updated independently without damaging the rest of the window contents.

You can see the opaque regions by using “Quartz Debug” and turning on the “Show opaque regions” checkbox. Sadly, screen-capture doesn’t appear to grab the yellow highlights you get when you turn on the “Flash screen updates” checkbox, so you’ll have to imagine that.

Opaque regions displayed in green

This is what all the major browsers are doing on macOS, and now we are too.

This also managed to simplify a lot of code in the macOS backend, which is always appreciated.

https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/4477

GSignalGroup and GBindingGroup

Some of the first abstractions we made when creating GNOME Builder are now available to everyone in GObject!

In particular, writing Text Editors requires tracking lots of information and changes from various sources. Sometimes those changes come from 2nd-degree objects via the object you really care about.

For example, with a GtkTextView that might mean tracking changes to a GtkTextBuffer, GtkTextTagTable, or many other application-specific accessory objects through the form of signals and properties.

To make that easier, we now have GSignalGroup and GBindingGroup merged directly into GObject. At their core, they allow you to treat a series of signals or bindings as a group and connect/disconnect them as a group. Furthermore, you can use g_object_bind_property() to bind the target instance to the binding/signal group removing even more code from your application. More than likely you’ll be better about supporting how things can change out from under you too.

In the process of upstreaming these objects, we did a bit of hardening, API cleanup, and thread-safety guarantees.

Now, go and delete some code!

Creating GtkSourceView style schemes

GtkSourceView has the concept of “style schemes” which map language features (types, keywords, strings, etc) to various colors and font properties. Creating them can be a bit laborious even if you’re starting with a color palette prepared. The artistic process is iterative so reducing the time between iterations is paramount.

Furthermore, I have aphantasia which means I need to be able to see things visually because I lack the wiring to simply imagine it in my head.

With that in mind, I spent a little time over the holiday season creating a standalone application to create or modify modify style-schemes. It provides live preview and targets the new features in GtkSourceView 5.x. It’s not a circle app or anything yet, but you can always grab it from the CI pipeline artifacts which include a Flatpak bundle.

It is a simple application with humble requirements.

  • Edit attributes about a style scheme.
  • Manage the color palette and allow importing color palettes from external formats (such as GIMP color palettes).
  • Visually modify styles for global, common, and per-language override.

My hope is that this encourages people to create art, even if you have neurological wiring similar to me.

Text Editor Happenings

Text Editor has really been shaping up in the past couple weeks as we race towards getting things ready for GNOME 42.

Preferences Dialog

A screenshot of the preferences dialog We removed the preferences sidebar experiment because it was a bit clunky and none of the other core apps shared the design metaphor. Instead we’ve brought back a preferences dialog, albeit with an improved design. It builds on the previous GtkSourceStyleSchemePreview work but with a filtered set based on the current light/dark desktop setting.

Open Popover

A screenshot of the files popover The “Open” popover also got another round of design work based on using it for a while. The style is a bit slimmer and more to the point. It is also much more keyboard navigable now. It’s nice to be able to hit Ctrl+K, type a few characters, and either hit Return to open the first match or navigate through the list with arrows. Escape will return you back to the editor. Easy stuff.

Spaces Drawing

I added a GSetting for those that rely on visual spaces. Even though there is no UI for this at the moment you can still get what you want fairly easily using the gsettings command line tool.

flatpak run --command=gsettings org.gnome.TextEditor.Devel \
    set org.gnome.TextEditor draw-spaces \
    "['space','tab','trailing']"

Vim

If you’re feeling experimental you can even try my easter egg to test out the Vim emulation using GtkSourceVimIMContext. It probably will never be an advertised feature, but it’s there.

flatpak run --command=gsettings org.gnome.TextEditor.Devel \
    set org.gnome.TextEditor keybindings vim # or "default"

Recoloring

The most visual piece of work this week was in introducing recoloring support. It builds atop libadwaita and uses a CSS provider to override the colors in the theme. I expect there will be a recoloring API in the not-too distant future for libadwaita which will provide this for us.

When you select a style-scheme Text Editor will now use the colors defined in the scheme to alter how the entire application looks. For example, below are screenshots for a number of style-schemes both bundled with GtkSourceView and found on the internet.

A screenshot of a style scheme A screenshot of a style scheme
A screenshot of a style scheme A screenshot of a style scheme
A screenshot of a style scheme A screenshot of a style scheme
A screenshot of a style scheme A screenshot of a style scheme
A screenshot of a style scheme

A screenshot of the preferences And yes, the coloring is reflected throughout the entire interface.

As always, I tend to be slow to blog but quick to drivel over here.

GtkSourceView is getting a VIM emulator

One of the things we spent some time on early with Builder was creating built-in VIM emulation. Originally it was written in C, but then got converted to this nasty CSS for a couple of reasons. First, that keybindings in GTK 3 were largely done with CSS due to how we moved away from gtkrc in the 2.x to 3.x migration. And beyond that, CSS was the one way that’d let us share a lot of keybinding code between shortcut themes so we could support emacs, vim, sublime, etc.

One thing that didn’t work well was VIM’s “modes”. People often think of VIM as a single state (normal, insert, replace, visual, etc). In reality it’s a stack of them. Just look at the following state diagram if you’re going to actually to match things somewhat accurately. Doing this in a manner that would allow sharing between shortcut themes in Builder was simply beyond the amount of time we had to spend on it given all the other features that needed to land.

As we start to port Builder to GTK 4 this is one area that worried me the most. GTK 4 has improved so much in this regard, but we also have this giant legacy ball of tar. So recently I started looking at a new approach to providing VIM emulation in a way that would let us improve the quality and extend it’s reach.

The experiment I’ve come up with so far is to build upon GtkIMContext. This allows us to filter key presses and releases from the widget and do so in a way that allows layered input-methods (ibus, emoji, unicode, etc) to continue working.

One question that will always pop up is “why not use $some_form_of_vim” and bridge that. The answer is pretty simple, it’s a huge dependency to take on as a library. It also requires synchronizing text and across boundaries which may not even be capable in places where GtkSourceView is used. Beyond that there are a large number of impedance mismatches in how things work from a generic standpoint.

But fear not, we can actually do pretty good for the most commonly used things. We can probably even implement registers (some of it is there already) and marks for people that want it. Some day I imagine we’ll land code folding upstream, so even supporting that might be possible.

My goal is to get this into 5.4 so that we have it well tested by time Builder lands a GTK 4 port. The exposed API is very small and just the single object GtkSourceVimIMContext. By keeping the implementation private we allow ourselves quite a bit of flexibility to adapt how we implement things if and when that is necessary.

I haven’t decided yet if we would want to have a property like GtkSourceView:enable-vim to simply enable it. I’m hesitant to do that because supporting it correctly can still be somewhat involved. You need presentation of a command bar and active command state for example. To do that requires application coordination anyway, so perhaps requiring the application to add a GtkEventControllerKey with the GtkSourceVimIMContext is the better option.

If you’d like to help on any of this, feel free to send me MRs. Everyone has slightly different usage of VIM and there are lots of missing spots you can fill in.


git clone https://gitlab.gnome.org/GNOME/gtksourceview.git
cd gtksourceview
git checkout wip/chergert/vim
meson build
cd build
ninja
./tests/test-vim
hack and repeat

As always, I usually leave screenshots and videos for people here if you can handle my other mindless dribble.

GtkSourceStyleSchemePreview

In the past, we had a style scheme chooser widget to help people find style schemes that look useful. It was fairly rudimentary and doesn’t fit how we’d like style scheme selection to work going forward. So in what will become GtkSourceView 5.4, I’ve added the GtkSourceStyleSchemePreview widget which you can use to preview a specific style scheme.

It allows you to create your own chooser like is seen in this screenshot of Text Editor.

As usual, you can try it for yourself with Text Editor Nightly and follow along for more timely updates here.

Private Flatpak installations in Builder

Builder needs to deal with many SDK and SDK extensions for applications built upon Flatpak.

One thing I never liked about how we did this up until now was that we needed to install Flatpak remotes into the user’s personal Flatpak installation. First, because we needed to add Flathub and gnome-nightly repositories. Secondly, once a year we need to add the flathub-beta remote due to post-branch SDKs relying on beta extensions.

Previously this would pollute things like GNOME Software with versions of applications that you might not care about as a user.

In Builder 41, a private Flatpak installation is used in $XDG_DATA_DIRS which contains those remotes. Additionally we set a filter to only allow runtimes and specifically ones matching certain globs.

Ignoring GtkTextTag when printing

Now that Text Editor has spell checking integrated I needed a way to print without displaying tags such our “misspelled word” underline squiggles. So GtkSourceView 5.2 will include gtk_source_print_compositor_ignore_tag() to do the obvious thing.

Previously, If you wanted to do this, you had to remove all your tags and then print, only to restore them afterwards. This should be a lot more convenient for people writing various GtkSourceView-based text editors. Although, I’m suspect many of them weren’t even doing this correctly to begin with, hence this PSA.

GtkSourceView Searching with PCRE2

Last year I did some work to make GtkSourceView use PCRE2 for syntax highlighting. The primary motivation there was to improve syntax highlighting performance by using PCRE2’s JIT capability.

However, that left us in an odd place with how GtkSourceSearchContext works for regex-enabled search. It was using GRegex which itself uses PCRE (1). It’s pretty clear that the goal is to completely deprecate GRegex in GLib and it’s days are numbered. In particular, there is a lot we can’t do to control the execution environment and protect against things like stack overflows. Worsening things, PCRE doesn’t appear to be maintained these days.

So I finally got around to moving GtkSourceSearchContext to using our very-GRegex-looking PCRE2 wrapper ImplRegex after fixing a number of bugs in it. However, it does not use the JIT because that seems a bit unnecessary for something meant to be interactive. We want latency from Regex creation to execution to be quite low.

It will be available in nightlies soon for the 5.x series. I’m hesitant to backport it to 4.x or 3.x because it’s a rather large subsystem change and I’d like to reduce chances of breaking things I can’t reasonably test. Given the relatively few GtkSourceView 5.x applications in the wild (other than my own) I feel it’s a more acceptable change there.