Introducing Bonsai

TL;DR: Pair your Linux devices, developer APIs to share files, create object graphs with partial sync between devices, transactions, secondary indexes, rebasing, and more built upon GVariant and LMDB. Tooling to build cloudless multi-device services.

A picture of a Bonsai tree and a gnomeI’ve been spending a great deal of time thinking about what types of products I’d like to see in GNOME and what is missing to make that happen.

One observation is that I want access to my files and application data on all my computing devices but I don’t want to store that data on other peoples computers. I have computers, they have internet access, I shouldn’t have to use a multi-tenancy cloud if I’m running as much Free Software as I do. But if that is going to be competitive it needs to be easier than the alternatives.

But to build this I need a few fundamental layers to build applications atop. I’ll need access to files using all the GIO file APIs we love (GFile, GFileEnumerator, GIOStream, etc). I’ll also need the ability to read and write application data in a way that can be shared between devices which may not always be connected to my home Wi-Fi. In particular, we need to give developers great tools to make applications that natively support device synchronization.

What I’ve built to experiment with this all is Bonsai. It is very much an experiment at this phase but it is getting interesting enough to collaborate with others who would like to join me.

Bonsai consists of a daemon that you run on your “mostly connected” computer. Although that could easily be a raspberry pi quality computer in your home. That computer hosts the “upstream” storage space for files and application content.

Other devices like laptops, phones, or IoT can be paired with that primary device. They communicate using TLS connections using pinned self-signed certificates with point-to-point D-Bus serialization on top. The D-Bus serialization makes it convenient to use gdbus-codegen to generate proxies and services.

One service available to devices is the storage service. It can be consumed from libbonsai-storage to allow applications to browse, create, move, modify and stream file content.

Applications are much better when they can communicate between devices. So a Data-Access-Object library, aptly named libbonsai-dao, provides serializable object storage built upon GVariant and LMDB. It supports primary and secondary indexes, queries, cursors, transactions, and incremental sync between devices. It has the ability to rebase local changes atop changes pulled from the primary Bonsai device.

That last bit is neat because it means if an application is running on two devices which create new content they don’t clobber each others history.
The primary issue here is dealing with merge conflicts but libbonsai-dao provides some design for data objects to do the right thing.

Bonsai could also could serve as a base to build interesting services like backup, VPN, media sharing and casting, news, notes, calendars, contacts, and more. But honestly, it can only do that if people are actually interested in something like this. If so, let me know and see if you can lend your time or ideas for what you’d want this to become.

Sysprof Updates

I just uploaded the sysprof-3.33.4 tarball as we progress towards 3.34. This alpha release has some interesting new features that some of you may find interesting as you continue your quests to improve the performance of your system by improving the software running upon it.

An image of Sysprof with various performance graphs

For a while, I’ve been wondering about various ways to move GtkTextView forward in GTK 4. It’s of particular interest to me because I spent some time in the GTK 3 days making it faster for smooth scrolling. However, the designs that were employed there work better on the traditional Xorg setup than they do on GTK 3’s Wayland backend. Now that GTK 4 can have a proper GL pipeline, there is a lot of room for improvement.

Thanks to the West Coast Hackfest, I had a chance to sit down with Matthias and work through that design. GtkLabel was already using some accelerated text rendering so we started by making that work for GtkTextView. Then we extended the GSK PangoRenderer to handle the rest of the needs of GtkTextView and Matthias re-implemented some features to avoid cairo fallbacks.

After the hackfest I also found time to implement layout caching of PangoLayout. It helps reduce some of the CPU overhead in calculating line layouts.

As we start using the GPU more it becomes increasingly important to keep the CPU usage low. If we don’t it’s very likely to raise overall energy usage. To help keep us honest, I’ve added some RAPL energy statistics to Sysprof.

Sysprof design work

Since my last post, I’ve been working on a redesign of Sysprof (among other things) to make it a bit more useful and friendly to newcomers.

Many years ago, I worked on a small profiler project called “Perfkit” that never really went anywhere. I had already done most of my UI research for this years ago, so it was pretty much just a matter of applying that design to the Sysprof code-base.

Now you can individually show extra detail rows and much more. Same great Sysprof 3-part callgraph breakdown.

A screenshot of Sysprof

I’ll get some delayed 3.33.3 tarballs out this week.

GTK 3 Frame Profiler

I back-ported the GTK 4 frame-profiler by Matthias to GTK 3 today. Here is an example of a JavaScript application using GJS and GTK 3. The data contains mixed native and JS stack-traces along with compositor frame information from gnome-shell.

What is going on here is that we have binary streams (although usually captured into a memfd) that come from various systems. GJS is delivered a FD via GJS_TRACE_FD environment variable. GTK is delivered a FD via GTK_TRACE_FD. We get data from GNOME Shell using the org.gnome.Sysprof3.Profiler D-Bus service exported on the org.gnome.Shell peer. Stack-traces come from GJS using SIGPROF and the SpiderMonkey profiler API. Native stack traces come from the Linux kernel’s perf_event_open system. Various data like CPU frequency and usage, memory and such come from /proc.

Muxing all the data streams uses sysprof_capture_writer_cat() which knows how to read data frames from supplemental captures and adjust counter IDs, JIT-mappings, and other file-specific data into a joined capture.

A quick reminder that we have a Platform Profiling Initiative in case you’re interested in helping out on this stuff.

Sysprof Developments

This week I spent a little time fixing up a number of integration points with Sysprof and our tooling.

The libsysprof-capture-3.a static library is now licensed under the BSD 2-clause plus patent to make things easier to consume from all sorts of libraries and applications.

We have a MR for GJS to switch to libsysprof-capture-3.a and improve plumbing so Sysprof can connect automatically.

We also have a number of patches for GLib and GTK that improve the chances we can get useful stack-traces when unwinding from the Linux kernel (which perf_event_open does).

A MR for GNOME Shell automatically connects the GJS profiler which is required as libgjs is being used as a library here. The previous GJS patches only wire things up when the gjs binary is used.

With that stuff in place, you can get quite a bit of data correlated now.

# Logout, Switch to VT2
sysprof-cli -c "gnome-shell --wayland --display-server" --gjs --gnome-shell my-capture.syscap

If you don’t want mixed SpiderMonkey and perf stack-traces, you can use --no-perf. You can’t really rely on sample rates between two systems at the same time anyway.

With that in place, you can start correlating more frame data.

Sysprof Developments

Earlier this month, Matthias and I teamed up to push through some of our profiling tooling for GTK and GNOME. We took the occasional work I had done on Sysprof over the past few years and integrated that into the GTK-4.x tree.

Sysprof uses a binary log file to store information about execution in a manner that is easy to write-buffer and read-back using positioned reads. It helps keep the sampling overhead of sysprof low. But it’s too detail oriented for each application supporting the format to write. To make this stuff reusable I created a libsysprof-capture-3.a static library we embed from various layers of the platform.

GTK-4.x is now using this. Builder itself uses it to log internal statistics, tracing data, and counters for troubleshooting. I’ve also put forward patches for GJS to integrate with it. Georges revamped and pushed forward a prototype by Jonas to integrate with Mutter/Shell and get us frame timings and Cogl pipeline data. With some work we can finish off the i915 data sources that Eric Anholt did to correlate GPU commands too.

What this means for developers is that soon we’ll be able to capture system information from various layers in the stack and correlate them using similar clocks. We’re only scratching the surface right now, but it’s definitely promising. It’s already useful to quantify the true performance improvements of merge-requests in Mutter and Shell.

To help achieve this goal during the 3.34 cycle, I’ve started the GNOME Profiling Initiative to track integration of various problem spaces. If you’re an application developer, you can use sysprof_capture_writer_new_for_env() to create a new SysprofCaptureWriter if Sysprof is profiling you (otherwise you’ll get NULL back). Use that to write marks, logs, metadata, files, or anything else you would like to capture.

If you’re interested in helping to write more data collectors, that would be appreciated. Data sources like battery voltage/wattage consumption seem like good candidates so that we can better optimize our platform for battery-based devices.

I have a Sysprof Copr repository for Rawhide and F30 if you’d like to try stuff out and submit issues.

Many thanks to Red Hat for sponsoring all the work I do on GNOME and my amazing manager Matthias for visiting Portland to make this stuff happen even sooner.

As always, follow my grumpy ramblings on birdsite for early previews of my work.

Improving the Container Workflow

As I mentioned in my talk at Scale 17x, if you aren’t using containers for building your application yet, it’s likely you will in the not-so-distant future. Moving towards an immutable base OS is a very likely future because the security advantages are so compelling. With that, comes a need for a mutable playground, and containers generally fit the bill. This is something I saw coming when I started making Builder so we have had a number of container abstractions for some time.

With growing distributions, like Fedora’s freshly released Silverblue, it becomes even more necessary to push those container boundaries soon.

This week I started playing with some new ideas for a terminal workspace in Builder. The goal is to be a bit of a swiss-army knife for container oriented development. I think there is a lot we can offer by building on the rest of Builder, even if you’re primary programming workhorse is a terminal and the IDE experience is not for you. Plumbing is plumbing is plumbing.

I’m interested in getting feedback on how developers are using containers to do their development. If that is something that you’re interested in sharing with me, send me an email (details here) that is as concise as possible. It will help me find the common denominators for good abstractions.

What I find neat, is that using the abstractions in Builder, you can make a container-focused terminal in a couple of hours of tinkering.

I also started looking into Vagrant integration and now the basics work. But we’ll have to introduce a few hooks into the build pipeline based on how projects want to be compiled. In most cases, it seems the VMs are used to push the app (and less about compiling) with dynamic languages, but I have no idea how pervasive that is. I’m curious how projects that are compiling in the VM/container deal with synchronizing code.

Another refactor that needs to be done is to give plugins insight into whether or not they can pass file-descriptors to the target. Passing a FD over SSH is not possible (although in some cases can be emulated) so plugins like gdb will have to come up with alternate mechanisms in that scenario.

I will say that trying out vagrant has been a fairly disappointing experience compared to our Flatpak workflow in Builder. The number of ways it can break is underwhelming and the performance reminds me of my days working on virtualization technology. It makes me feel even more confident in the Flatpak architecture for desktop developer tooling.

Translucent Completion

Sometimes completion windows get in the way of reading the surrounding code. With Builder Nightly, you can press and release Left Control to toggle translucency of the completion window.

Flatpaking Terminals

One thing Builder has done for a long time is make terminals work seamlessly even if distributed using container technologies. Because pseudo-terminals are steeped in esoteric UNIX history, it can be non-obvious how to make this work.

I’m in a place to help you not have to deal with that pain because I’ve already gone through it. So I created some utility code and a demo application that can be packaged with Flatpak. If it detects it’s running under Flatpak it will use a few techniques to get a user-preferred shell executed on the host with a PTY controlled by application.

Check out the code.

Edit: The flatterm repository has been updated to use the brand new VTE_PTY_NO_CTTY flag that was added in response to this blog post. Users of Vte from git (what will be 0.58) get to enjoy writing even less code.