Faster Numbers

The venerable GtkSourceView project provides a GtkWidget for various code languages. It has a number of features including the most basic, showing a line number next to your line of text.

A screenshot of GNOME Text Editor with line numbers enabled containing the file gtktextbuffer.c.

It turns out that takes a lot more effort than you might think, particularly when you want to do it at 240hz with kinetic scrolling on crappy hardware that may barely have enough engine for the GL driver.

First, you need to have the line number as a string to be rendered. For a few years now, GtkSourceView has code which will optimizes the translation from number to strings with minimal overhead. If you g_snprintf(), you’re gonna be slow.

After that you need to know the X,Y coordinate of the particular line within the gutter and it’s line height when wrapped. Then you need to know the measured pixel width of the line number string. Further still you need the xalign/yalign and xpad/ypad to apply proper alignments based on application needs. You may even want to align based on first line, last wrapped line, or the entire cell.

In the GtkSourceView 5.x port I created GtkSourceGutterLines which can cache some of that information. It’s still extremely expensive to calculate but at least we only have to do it once per-frame now no matter how many GtkSourceGutterRenderer are packed into the GtkSourceGutter.

After that, we can create (well recycle) a PangoLayout to setup what we want to render. Except, that is also extremely expensive because you need to measure the contents and go through a PangoRenderer for each line you render.

If you are kinetic scrolling through a GtkSourceView with something like a touch pad there is a good chance that a decent chunk of CPU is wasted on line numbers. Nicht gut.

Astute readers will remember that I spent a little time making VTE render faster this cycle and one of the ways to do that was to avoid PangoLayout. We can do the same here as it’s extremely simple and controlled input. Just cache the PangoGlyphInfo for 0..9 and use that to build a suitable PangoGlyphString. Armed with a PangoFont and said string, we can use gsk_text_node_new() and gtk_snapshot_append_node() instead of gtk_snapshot_render_layout().

A quick hour or so later I have given you back double digit CPU percentages but more importantly, smoother and lower latency input.

Sysprof makes it easy to locate, triage, and verify performance fixes.

A flamegraph showing that the line number gutter renderer in GtkSourceView was an extremely complex code path.

A flamegraph showing that line number rendering is now a very simple code path.

That said, in the future, if I were redesigning something to replace all of this I’d probably just use widgets for each line number and recycle them like GtkListView. Then you get GtkWidget render node caching for free. C’est la vie.

Builder GTK 4 Porting, Part IV

This week was a little slower as I was struggling with an adjustment to my new medication. Things progress nonetheless.

Text Editor

I spent a little time this week triaging some incoming Text Editor issues and feature requests. I’d really like this application to get into maintenance mode soon because I have plenty of other projects to maintain.

  • Added support for gnome-text-editor - to open a file from standard input, even if you’re communicating to a single instance application from terminal.
  • Branch GNOME 42 so we can add new strings.
  • Fix a no-data-loss crash during shutdown.

Template-GLib

  • Fix template evaluation on macOS.
  • Make boolean expression precedence more predictable.
  • Cleanup output of templates with regards to newlines.

libpanel

  • Propagate modified page state to tabs
  • Some action tweaks to make things more keyboard shortcut friendly.

Builder

  • Merged support for configuration editing from Georges.
  • Add lots of keybindings using our new keybinding engine.
  • Track down and triage that shortcut controllers do not capture/bubble to popovers. Added workarounds for common popovers in Builder.
  • Teach Builder to load keybindings from plugins again and auto-manage them.
  • Lots of tweaks to the debugger UI and where widgetry is placed.
  • Added syntax highlighting for debugger disassembly.
  • Added menus and toggles for various logging and debugger hooks. You can get a breakpoint on g_warning() or g_critical() by checking a box.
  • Ability to select a build target as the default build target finally.
  • More menuing fixes all over the place, particularly with treeviews and sourceviews.
  • Fix keyboard navigation and activation for the new symbol-tree
  • Port the find-other-file plugin to the new workspace design which no longer requires using global search.
  • GTK 4 doesn’t appear to scroll to cells in textview as reliably as I’d like, so I dropped the animation code in Builder and we jump strait to the target before showing popovers.
  • Various work on per-language settings overrides by project.
  • Drop the Rust rls plugin as we can pretty much just rely on rust-analyzer now.
  • Lots of CSS tweaks to make things fit a bit better with upcoming GNOME styling.
  • Fix broken dialog which prevented SDK updates from occurring with other dependencies.

A screenshot of builder's find-other-file plugin

A screenshot of Builder's debugger

A screenshot showing the build target selection dialog

A screenshot of the run menu A screenshot of the logging menu

Builder GTK 4 Porting, Part II

Another week of work towards porting Builder to GTK 4. Since I can’t add to TWIG from IRC, I’ll try harder to drop some occasional updates here.

GtkSourceView

  • Merged fixes for highlighting unicode literals for C
  • Improved parsing of language values in snippet bundles
  • GtkSourceGutter will not correctly mark prelit and selection quarks within GtkSourceGutterLines.
  • Fixed a bunch of little mouse pointer annoyances when using GtkSourceHover interactive tooltips.
  • GtkSourceGutterRenderers can now opt-out of signal emission for GtkSourceGutterRenderer::query_data(). Signal emission with GObject is rather slow, so avoiding it on every line can be helpful. Just set the virtual method pointer to NULL. The signal was only ever added to make porting easier anyway.

libpanel

  • Merged fixes to be used as a subproject with static libraries only
  • CSS styling matches recent changes in libadwaita, particularly around making navigation tabs and panel frame headers more unified.
  • PanelWidget no longer uses a GtkBinLayout so that it’s easier for subclasses to integrate with popovers from size_allocate() to call gtk_popover_present().

Builder

  • Georges did a live coding stream where they ported a bunch of the “buildui” plugin. That is merged to the GTK 4 port now. It brings a number of features back to the UI including the build terminal, run terminal, build panel (with pipeline stages, warnings, and errors), and project information in the omnibar popover.
  • Günther did a bunch of work porting our old snippet files to the new XML-based snippet bundle format upstream in GtkSourceView. Along with that came porting of the snippets plugin for Builder’s new GTK 4-based editor.
  • Workspace windows have a bit better predictability when restoring sizes.
  • The project creation workflow was ported, albeit needs to have the redesign implemented still.
  • Lots of resiliency fixes for clang and symbol-tree plugins to improve life-cycle management.
  • The Valgrind plugin was ported to C. This was also back-ported to main because it fixed a number of oddities (crashes) occurring in PyGObject.
  • Builder’s “omni-gutter” GtkSourceGutterRenderer was ported to GTK 4 and got a lot of cleanups along the way. I believe there is still some outstanding things to fix such as handling rendering for symbolic icons as I’m pretty sure that’s not correct yet.
  • The “debuggerui” plugin has been ported to GTK 4 and appears to be working well now. This plugin is responsible for bridging the internal IdeDebugger interfaces to the UI interface.
  • Style schemes were updated for GtkSourceView 5
  • Tabs are now the default navigation interface for pages. There is likely still a lot to do around how we want empty frames to look and behave though.
  • The project-tree is now most ported, and with some workarounds to allow making GtkPopover work you can actually display popovers and activate menu items.
  • A long-standing plugin ordering issue has been fixed so that gtk/menus.ui embedded in plugin resources have menu-merging in the proper order.
  • Some incremental work landed to support per-project configuration of languages.
  • The “ls” plugin (directory views) supports “humanized” timestamps again and columns are resizable once more.
  • You can place panels in the right-side-bar now.

Upcoming

There are a bunch of foundational things to still get landed before I feel I can get Builder flipped over to our Nightly builds. In particular we need to land support for things like:

  • Keyboard shortcuts using GtkShortcutController. This was all done with libdazzle previously.
  • Allow plugins to define custom keyboard shortcuts and merge them into the controller.
  • Port “shellcmd” plugin and integrate keyboard shortcuts to apply those commands.
  • Finish rewrite of the search popover. I’m trying to delay this until GtkListView with sections is supported, as it would provide us a much greater path for performance.
  • A lot of our problems would be simpler if we could make GtkActionMuxer use an alternate action muxer parent from another (non-descendant) widget.
  • Configuration editing in the project configuration window. This is a new design so the port is not 1:1.

A screenshot of Builder with the project-tree context menu and debugger on display

A screenshot of Builder with various panels moved around to non-standard locations

Creating GtkSourceView style schemes

GtkSourceView has the concept of “style schemes” which map language features (types, keywords, strings, etc) to various colors and font properties. Creating them can be a bit laborious even if you’re starting with a color palette prepared. The artistic process is iterative so reducing the time between iterations is paramount.

Furthermore, I have aphantasia which means I need to be able to see things visually because I lack the wiring to simply imagine it in my head.

With that in mind, I spent a little time over the holiday season creating a standalone application to create or modify modify style-schemes. It provides live preview and targets the new features in GtkSourceView 5.x. It’s not a circle app or anything yet, but you can always grab it from the CI pipeline artifacts which include a Flatpak bundle.

It is a simple application with humble requirements.

  • Edit attributes about a style scheme.
  • Manage the color palette and allow importing color palettes from external formats (such as GIMP color palettes).
  • Visually modify styles for global, common, and per-language override.

My hope is that this encourages people to create art, even if you have neurological wiring similar to me.

GtkSourceStyleSchemePreview

In the past, we had a style scheme chooser widget to help people find style schemes that look useful. It was fairly rudimentary and doesn’t fit how we’d like style scheme selection to work going forward. So in what will become GtkSourceView 5.4, I’ve added the GtkSourceStyleSchemePreview widget which you can use to preview a specific style scheme.

It allows you to create your own chooser like is seen in this screenshot of Text Editor.

As usual, you can try it for yourself with Text Editor Nightly and follow along for more timely updates here.

Ignoring GtkTextTag when printing

Now that Text Editor has spell checking integrated I needed a way to print without displaying tags such our “misspelled word” underline squiggles. So GtkSourceView 5.2 will include gtk_source_print_compositor_ignore_tag() to do the obvious thing.

Previously, If you wanted to do this, you had to remove all your tags and then print, only to restore them afterwards. This should be a lot more convenient for people writing various GtkSourceView-based text editors. Although, I’m suspect many of them weren’t even doing this correctly to begin with, hence this PSA.

GtkSourceView Searching with PCRE2

Last year I did some work to make GtkSourceView use PCRE2 for syntax highlighting. The primary motivation there was to improve syntax highlighting performance by using PCRE2’s JIT capability.

However, that left us in an odd place with how GtkSourceSearchContext works for regex-enabled search. It was using GRegex which itself uses PCRE (1). It’s pretty clear that the goal is to completely deprecate GRegex in GLib and it’s days are numbered. In particular, there is a lot we can’t do to control the execution environment and protect against things like stack overflows. Worsening things, PCRE doesn’t appear to be maintained these days.

So I finally got around to moving GtkSourceSearchContext to using our very-GRegex-looking PCRE2 wrapper ImplRegex after fixing a number of bugs in it. However, it does not use the JIT because that seems a bit unnecessary for something meant to be interactive. We want latency from Regex creation to execution to be quite low.

It will be available in nightlies soon for the 5.x series. I’m hesitant to backport it to 4.x or 3.x because it’s a rather large subsystem change and I’d like to reduce chances of breaking things I can’t reasonably test. Given the relatively few GtkSourceView 5.x applications in the wild (other than my own) I feel it’s a more acceptable change there.

GtkSourceView Interactive Tooltips

During the past years (and especially last cycle) I’ve worked to push a number of features upstream from Builder into GtkSourceView. Not only does this improve the ecosystem for all applications, but it reduces the number of things we need to maintain downstream in Builder as we move to GTK 4.

That included a new completion engine, a new snippet engine with tooltips based on expansion points, updated gutter and gutter renderer designs, the editor overview map, background patterns, sysprof tracing integration, and most recently interactive tooltips.

You can read our nightly generated documentation to learn about it. In particular, GtkSourceHover is attached to the GtkSourceView and can have GtkSourceHoverProviders registered with it. Implementing that interface will be familiar to those who’ve implemented completion providers, as they work in a similar way.

This came from Builder where we have interactive tooltips for things like breakpoints, documentation, and symbol information. Other applications using GtkSourceView may find it useful when implementing Language Server Protocol’s hover providers.

A GTK 4 based Text Editor

It started as an application for me to verify the correctness of the GtkSourceView 5 API (which targets GTK 4). After that it helped me implement JIT support for GtkSourceView languages. Once that was done it became my test case while I wrote the GTK 4 macOS backend and revamped the GL renderer.

It is a simple and humble text editor. It does not have all the corner cases you’d expect from a text editor yet. It does not have aspirations to be a programmers text editor.

Now that you know this is very much a technology preview release only, you might be tempted to keep your important data away from it.

What it can do

  • Simple interface designed by the GNOME design team. You can find the mockups in the traditional places
  • Search and Replace
  • Typical GtkSourceView features
  • Quick access to recent documents
  • Multiple windows
  • Automatic discovery of .editorconfig and modelines settings
  • Light and dark mode
  • Automatically save files to drafts, restored in case of crash
  • Printing
  • Can be run from within a Flatpak sandbox and uses document portal for access to host files

What it cannot do

  • It doesn’t protect you from trying to open really large files
  • Support your custom GTK 4 theme
  • Auto-completion or snippets
  • Plugins
  • Custom file encodings
  • Spell check
  • Change style schemes beyond light and dark
  • Translations or Help of any kind

Building

Here is a release tarball.

If you’d like to test it out, one way is to clone the repository from GNOME Builder and click Run. Additionally, you can find a Flatpak in the gnome-nightly Flatpak repository.

GtkSourceView gets a JIT

I just merged a new regex implementation for GtkSourceView’s language specifications. Previously it used GRegex (based on PCRE) and now it uses PCRE2 directly similar to what VTE did.

Not only does this get us on a more modern PCRE implementation, but it also allows us to use new features such as a JIT.

JITs are interesting in that you can trade a little bit of memory and time to generate executable code upfront for huge gains in execution time. Given that you only compile language specifications once per regex, but execute them many, many times, it’s a worthwhile feature for GtkSoureView.

Trying to highlight the minified HTML of google.com/ won’t even highlight (due to timeouts) with GRegex. But with PCRE2 and the JIT, it can get by.

In many cases, I found that the cost to JIT was about 4x vs PCRE2 without JIT. For execution times it is about 4x reduction to use the JIT (but sometimes many times faster than that). When you run these regexes millions of times across an edited file, it can really cut down on the amount of energy consumed as well as time taken away from doing things like rendering GTK’s scene graph.

I should note that it’s about 4x improvement on a per-regex basis, so when you run potentially thousands of those in one main loop cycle, the improvement can be much more drastic in what you can do.

If you have any issues with language specifications please let us know! It’s a very large change, so I wouldn’t be surprised if there is some fallout.

Regex Creation
Language Min (msec) Max (msec) Average (msec) # of Calls Notes
C .001 .104 .007 91 gtktreeview.c
C (with JIT) .004 .383 .031 91
Difference .003 .279 .024
CSS .001 .711 .022 197 gtk-contained.css
CSS (with JIT) .004 3.147 .101 198
Difference .003 2.436 .079
Regex Execution Loading File
Language Min (msec) Max (msec) Average (msec) # Calls
C .000 .196 .003 17698 gtktreeview.c
C (with JIT) .000 .061 .001 17698
Difference -.000 -.135 -.002 ~35 ms
CSS .000 .211 .022 84347 gtk-contained.css
CSS (with JIT) .000 .061 .001 74812
Difference -.000 -.135 -.002 ~150 ms