Sysprof has gained network and disk device statistics. You can use the combined graphs for a quick overview, or view them individually.
Since my last post, I’ve been working on a redesign of Sysprof (among other things) to make it a bit more useful and friendly to newcomers.
Many years ago, I worked on a small profiler project called “Perfkit” that never really went anywhere. I had already done most of my UI research for this years ago, so it was pretty much just a matter of applying that design to the Sysprof code-base.
Now you can individually show extra detail rows and much more. Same great Sysprof 3-part callgraph breakdown.
I’ll get some delayed 3.33.3 tarballs out this week.
What is going on here is that we have binary streams (although usually captured into a memfd) that come from various systems. GJS is delivered a FD via
GJS_TRACE_FD environment variable. GTK is delivered a FD via
GTK_TRACE_FD. We get data from GNOME Shell using the
org.gnome.Sysprof3.Profiler D-Bus service exported on the
org.gnome.Shell peer. Stack-traces come from GJS using
SIGPROF and the SpiderMonkey profiler API. Native stack traces come from the Linux kernel’s
perf_event_open system. Various data like CPU frequency and usage, memory and such come from
Muxing all the data streams uses
sysprof_capture_writer_cat() which knows how to read data frames from supplemental captures and adjust counter IDs, JIT-mappings, and other file-specific data into a joined capture.
A quick reminder that we have a Platform Profiling Initiative in case you’re interested in helping out on this stuff.
This week I spent a little time fixing up a number of integration points with Sysprof and our tooling.
libsysprof-capture-3.a static library is now licensed under the BSD 2-clause plus patent to make things easier to consume from all sorts of libraries and applications.
We have a MR for GJS to switch to
libsysprof-capture-3.a and improve plumbing so Sysprof can connect automatically.
A MR for GNOME Shell automatically connects the GJS profiler which is required as
libgjs is being used as a library here. The previous GJS patches only wire things up when the
gjs binary is used.
With that stuff in place, you can get quite a bit of data correlated now.
# Logout, Switch to VT2 sysprof-cli -c "gnome-shell --wayland --display-server" --gjs --gnome-shell my-capture.syscap
If you don’t want mixed SpiderMonkey and perf stack-traces, you can use
--no-perf. You can’t really rely on sample rates between two systems at the same time anyway.
With that in place, you can start correlating more frame data.
Earlier this month, Matthias and I teamed up to push through some of our profiling tooling for GTK and GNOME. We took the occasional work I had done on Sysprof over the past few years and integrated that into the GTK-4.x tree.
Sysprof uses a binary log file to store information about execution in a manner that is easy to write-buffer and read-back using positioned reads. It helps keep the sampling overhead of sysprof low. But it’s too detail oriented for each application supporting the format to write. To make this stuff reusable I created a
libsysprof-capture-3.a static library we embed from various layers of the platform.
GTK-4.x is now using this. Builder itself uses it to log internal statistics, tracing data, and counters for troubleshooting. I’ve also put forward patches for GJS to integrate with it. Georges revamped and pushed forward a prototype by Jonas to integrate with Mutter/Shell and get us frame timings and Cogl pipeline data. With some work we can finish off the i915 data sources that Eric Anholt did to correlate GPU commands too.
What this means for developers is that soon we’ll be able to capture system information from various layers in the stack and correlate them using similar clocks. We’re only scratching the surface right now, but it’s definitely promising. It’s already useful to quantify the true performance improvements of merge-requests in Mutter and Shell.
To help achieve this goal during the 3.34 cycle, I’ve started the GNOME Profiling Initiative to track integration of various problem spaces. If you’re an application developer, you can use
sysprof_capture_writer_new_for_env() to create a new
SysprofCaptureWriter if Sysprof is profiling you (otherwise you’ll get
NULL back). Use that to write marks, logs, metadata, files, or anything else you would like to capture.
If you’re interested in helping to write more data collectors, that would be appreciated. Data sources like battery voltage/wattage consumption seem like good candidates so that we can better optimize our platform for battery-based devices.
Many thanks to Red Hat for sponsoring all the work I do on GNOME and my amazing manager Matthias for visiting Portland to make this stuff happen even sooner.
As always, follow my grumpy ramblings on birdsite for early previews of my work.
As I mentioned in my talk at Scale 17x, if you aren’t using containers for building your application yet, it’s likely you will in the not-so-distant future. Moving towards an immutable base OS is a very likely future because the security advantages are so compelling. With that, comes a need for a mutable playground, and containers generally fit the bill. This is something I saw coming when I started making Builder so we have had a number of container abstractions for some time.
With growing distributions, like Fedora’s freshly released Silverblue, it becomes even more necessary to push those container boundaries soon.
This week I started playing with some new ideas for a terminal workspace in Builder. The goal is to be a bit of a swiss-army knife for container oriented development. I think there is a lot we can offer by building on the rest of Builder, even if you’re primary programming workhorse is a terminal and the IDE experience is not for you. Plumbing is plumbing is plumbing.
I’m interested in getting feedback on how developers are using containers to do their development. If that is something that you’re interested in sharing with me, send me an email (details here) that is as concise as possible. It will help me find the common denominators for good abstractions.
What I find neat, is that using the abstractions in Builder, you can make a container-focused terminal in a couple of hours of tinkering.
I also started looking into Vagrant integration and now the basics work. But we’ll have to introduce a few hooks into the build pipeline based on how projects want to be compiled. In most cases, it seems the VMs are used to push the app (and less about compiling) with dynamic languages, but I have no idea how pervasive that is. I’m curious how projects that are compiling in the VM/container deal with synchronizing code.
Another refactor that needs to be done is to give plugins insight into whether or not they can pass file-descriptors to the target. Passing a FD over SSH is not possible (although in some cases can be emulated) so plugins like gdb will have to come up with alternate mechanisms in that scenario.
I will say that trying out vagrant has been a fairly disappointing experience compared to our Flatpak workflow in Builder. The number of ways it can break is underwhelming and the performance reminds me of my days working on virtualization technology. It makes me feel even more confident in the Flatpak architecture for desktop developer tooling.
Sometimes completion windows get in the way of reading the surrounding code. With Builder Nightly, you can press and release
Left Control to toggle translucency of the completion window.
One thing Builder has done for a long time is make terminals work seamlessly even if distributed using container technologies. Because pseudo-terminals are steeped in esoteric UNIX history, it can be non-obvious how to make this work.
I’m in a place to help you not have to deal with that pain because I’ve already gone through it. So I created some utility code and a demo application that can be packaged with Flatpak. If it detects it’s running under Flatpak it will use a few techniques to get a user-preferred shell executed on the host with a PTY controlled by application.
Edit: The flatterm repository has been updated to use the brand new
VTE_PTY_NO_CTTY flag that was added in response to this blog post. Users of Vte from git (what will be 0.58) get to enjoy writing even less code.
Our first 3.33 release has landed as we move towards 3.34. There is a lot to do this cycle in case you’re interested in contributing. The best way to get started is to dive into the code. We can help you with that on IRC.
Lots of this release is code behind the scenes, so screenshots won’t do them justice. But there are some visible goodies too.
We got a D-Bus Inspector inspired by D-feet. The long term goal is to merge that new code into D-feet itself.
Occasionally we got issues filed about not being able to half-tile Builder. The smaller your screen size, the more unrealistic of an expectation that is, but we can certainly do a little better. The new search box takes up less space and animates in as necessary.
We gained some initial Podman support so long as your system has support for
podman exec --preserve-fds. You can even debug inside the containers if they have
gdb in the container.
Some users were surprised by how Builder auto-downloaded SDKs and Runtimes to build projects. So we’ve added a confirmation dialog to that process so the user can make informed decisions.
I’ve also moved Git integration out of process. Both the Git and Flatpak plugins previously used git in process which had a number of drawbacks. We created some new API and the
gnome-builder-git daemon which provides access to common git operations using libgit2-glib and D-Bus serialization binary format over
stdout. We have push, stage, and signed commit support, but that currently lacks UI in this release.
The master branch of GtkSourceView (what will become 4.4) has moved to meson for development. I branched gtksourceview-4-2 for patch releases of 4.2.x which will remain autotools. Today’s release of gtksourceview-4.3.1 contains both autotools and meson. However 4.3.2 will remove autotools entirely.
I also landed some code to speed up line number drawing which was a non-trivial amount of the render cost while kinetic scrolling.