Sometimes completion windows get in the way of reading the surrounding code. With Builder Nightly, you can press and release Left Control
to toggle translucency of the completion window.
Month: April 2019
Flatpaking Terminals
One thing Builder has done for a long time is make terminals work seamlessly even if distributed using container technologies. Because pseudo-terminals are steeped in esoteric UNIX history, it can be non-obvious how to make this work.
I’m in a place to help you not have to deal with that pain because I’ve already gone through it. So I created some utility code and a demo application that can be packaged with Flatpak. If it detects it’s running under Flatpak it will use a few techniques to get a user-preferred shell executed on the host with a PTY controlled by application.
Edit: The flatterm repository has been updated to use the brand new VTE_PTY_NO_CTTY
flag that was added in response to this blog post. Users of Vte from git (what will be 0.58) get to enjoy writing even less code.
Builder 3.33.1
Our first 3.33 release has landed as we move towards 3.34. There is a lot to do this cycle in case you’re interested in contributing. The best way to get started is to dive into the code. We can help you with that on IRC.
Lots of this release is code behind the scenes, so screenshots won’t do them justice. But there are some visible goodies too.
We got a D-Bus Inspector inspired by D-feet. The long term goal is to merge that new code into D-feet itself.
Occasionally we got issues filed about not being able to half-tile Builder. The smaller your screen size, the more unrealistic of an expectation that is, but we can certainly do a little better. The new search box takes up less space and animates in as necessary.
We gained some initial Podman support so long as your system has support for podman exec --preserve-fds
. You can even debug inside the containers if they have gdb
in the container.
Some users were surprised by how Builder auto-downloaded SDKs and Runtimes to build projects. So we’ve added a confirmation dialog to that process so the user can make informed decisions.
I’ve also moved Git integration out of process. Both the Git and Flatpak plugins previously used git in process which had a number of drawbacks. We created some new API and the gnome-builder-git
daemon which provides access to common git operations using libgit2-glib and D-Bus serialization binary format over stdin
/stdout
. We have push, stage, and signed commit support, but that currently lacks UI in this release.
GtkSourceView moved to Meson
The master branch of GtkSourceView (what will become 4.4) has moved to meson for development. I branched gtksourceview-4-2 for patch releases of 4.2.x which will remain autotools. Today’s release of gtksourceview-4.3.1 contains both autotools and meson. However 4.3.2 will remove autotools entirely.
I also landed some code to speed up line number drawing which was a non-trivial amount of the render cost while kinetic scrolling.
Podman Support in Builder
For years now, Builder has had rich abstractions for containers built right into the core of the IDE. Builder even knows the difference between build and runtime containers which naturally maps with Flatpak SDKs like org.gnome.Sdk
vs org.gnome.Platform
.
With the advent of operating systems focused on immutability, like Fedora Silverblue, developers are going to be increasingly developing in containers.
The technology underlying projects like Toolbox is podman. It provides a command-line tool to manage containers without a daemon by using the various container APIs afforded to us in modern Linux.
Bridging Builder’s container APIs to support podman was pretty painless on my part. A couple hours to choose the right abstractions and implementing them led me to a missing piece in podman; passing FDs to the container.
The reason that Builder requires this is that we often need to communicate across containers. An easy way to do that is over a pair of pipe()
since it is anonymous. By anonymous, I mean we don’t need to share any file-system hierarchy, IPC or network namespaces, or even PTY namespace.
The most important piece that requires this in Builder is our GDB-based debugger. We use GDB inside the container so it has native access to things like build sources, libraries, symbols, and more. This is all orchestrated using GDB’s mi2
interface over a PTY, with a second PTY for the target process. When GDB lands on a breakpoint, we know how to translate paths between Builder’s container (usually Flatpak) and the target container (in this case, podman). Doing so ensures that we open the right file and line number to the user. Fundamentals, of course.
So a couple weeks later and podman exec
has gained the --preserve-fds=N
option, available on Rawhide and Fedora 30 (currently in updates-testing). If you have all the necessary bits in place, Builder will allow you to select a podman container from Build Preferences, and you’re off to the races.
Furthermore, you can even seamlessly get a terminal in the build environment with Control+Alt+Shift+T
which can prove useful if you have to install dependencies.
Since we don’t know much about the container, we don’t have the ability to install dependencies on your behalf. But if someone were to work on Dockerfile support, I don’t see that as an intractable problem.
Here is a quick test command-line program debugging in Builder using the GDB backend to prove it all works.
Designing for Sandboxes
One of the things I talked about in my talk at Scale 17x is that there are a number of platform features coming that are enevitable.
One of those is application sandboxing.
But not every component within an application is created equal or deserves equal access to user data and system configuration. Building the next big application is increasingly requiring thinking about how you segment applications into security domains.
Given the constraints of our current operating systems, that generally means processes. Google’s Chrome was one of the first major applications to do this. The Chrome team had created a series of processes focused on different features. Each of those processes had capabilities removed (such as network, or GPU access) from the process space to reduce the damage of an attack.
Recently Google released sandboxed-api, which is an interesting idea around automatically sandboxing libraries on Linux. While interesting, limiting myself to designs that are Linux only is not currently realistic for my projects.
Since I happen to work on an IDE, one of the technologies I’ve had to become familiar with is Microsoft’s Language Server Protocol. It’s a design for worker processes to provide language-specific features.
It usually works like this:
- Spawn a worker process, with a set of pipe()s for stdin/stdout you control
- Use JSONRPC over the pipe()s with some well-formatted JSON commands
This design can be good for sandboxing because it allows you to spawn subprocesses that have reduced system capabilities, easily clean up after them, and provides an IPC format. Despite having written jsonrpc-glib and a number of helpers to make writing JSON from C rather clean, I’m still unhappy with it for a number of reasons. Those reasons range from everything from performance to correctness to brittleness of nonconforming implementations.
I’d like to use this design in more than just Builder but those applications are more demanding. They require passing FDs across the process boundary. (Also I’m sick of hand writing JSON RPCs and I don’t want to do that anymore).
Thankfully, we’ve had this great RPC system for years that fits the bill if you reuse the serialization format: DBus.
- No ties to a DBus daemon
- GDBus in GLib has a full implementation that plays well with async/sync code
- gdbus-codegen can generate our RPC stubs and proxies
- Well defined interfaces in XML files
- Generated code does type enforcement to ensure contracts
- We can easily pass FDs across the process boundary, useful for memfd/tmpfs/shm
To setup the sandboxes, we can use tools like flatpak-spawn
or bwrap
on Linux to restrict process capabilities before launching the target process. Stdin/stdout is left untouched so that we can communicate with the subprocess even after capabilities are dropped.
Before I (re)settled on DBus, I tried a number of other prototypes. That included writing an interface language/codegen for JSONRPC, using libvarlink, Thrift’s c_glib compiler and protobufs. I’m actually surprised I was happiest with the DBus implementation, but that’s how it goes sometimes.
While I don’t expect a lot of sandboxing around our Git support in Builder, I did use it as an opportunity to prototype what this multi-process design looks like. If you’re interested in checking it out, you can find the worker sources here.
What excites me about the future is how this type of design could be used to sandbox image loaders like GdkPixbuf. One could quite trivially have an RPC that passes a sealed memfd for compressed image contents and returns a memfd for the decoded framebuffer or pre-compressed GPU textures. Keep that process around a little while to avoid fork()
/exec()
overhead, and we gain a bit of robustness with very little performance drawbacks.