Introducing deviced

Over the past couple of weeks I’ve been heads down working on a new tool along with Patrick Griffis. The purpose of this tool is to make it easier to integrate IDEs and other tooling with GNU-based gadgets like phones, tablets, infotainment, and IoT devices.

Years ago I was working on a GNOME-integrated home router with davidz which sadly we never finished. One thing that was obvious to me in that moment of time was that I will not do another large scale project until I have better tooling. That was Builder’s genesis and device integration is what will make it truly useful to myself and others who love playing with GNU-friendly gadgets.

Now, building an IDE is a long process. There is a ton of code to write, trade-offs to work through, and persistence beyond what any reasonable programmer would voluntarily sign up for. But the ends justify the slog.

So what we’ve created is uninterestingly called “deviced”. It currently has three components. A deviced daemon lives on the target device that we’re interested in writing software for. A GObject-based libdeviced library provides access to discover and connect to devices and do interesting things on them. Lastly, devicectl is a readline-based command line tool that allows you to interact with these devices without having to write a program using libdeviced.

The APIs in libdeviced are appropriately abstracted so that we can provide different transports in the future. Currently, we only have network-based communication but we will implement a USB transport in the not-too-distant future. Other protocols such as SSH or custom micro-controllers can be added. Although something like SSH is more complex because it’d be the combination of both a protocol and how to run commands to get the intended effect, which is non-portable. It will be possible to support devices that do not run deviced, but that is currently out of scope.

To allow devices to be discover-able, deviced will broadcast it’s presence using mDNS on networks it is configured to listen (based on network-manager connection UUID). Long term my goal is that you can configure deviced access in Control Center, similar to “Sharing and Privacy”. The network protocol is rather simple as it’s just JSON-RPC over TLS with self-signed certificates. When a client connects to the daemon, a gnome-shell notification is presented allowing you to accept the connection. At that point, the client certificate is saved for future validation.

Our libdeviced library is GObject introspectable and should therefore work with a number of languages.

Right now, only Flatpak applications are supported, but we have abstractions to allow for contributions to support additional application layers like docker or plain old .desktop files. Currently you can push flatpak applications and runtimes to the device and install them and run them. If you have a new enough Flatpak, you can do delta updates.

It can even bridge multiple PTY devices for a shell, which isn’t really meant to be an SSH replacement, but more of a single abstraction we can use to be able to control a debugger and inferior from the IDE tooling.

There are still lots of little bugs to shake out and more bits to implement, but this is a pretty sweet 2-week proof of concept.

Here is a 20 second demo running on a single machine. It’s the same when using multiple machines except you get the notification on the programmable device rather than on your workstation. Obviously for IoT devices we’d need to create some sort of freedesktop notification bridge or alternate notification mechanism.

Anytime you work on a new project people will inevitably ask “why not just use XYZ”. In this case, I would expect both SSH and ADB to fall into that category. Most importantly, libdeviced is going to be about providing a single “remote device” abstraction for us in Builder. So it’s reasonable that we could abstract both of those systems from libdeviced. But neither of those provide the work-flow I envision for out-of-box experience, hence the deviced daemon. In the ADB case, it will be very difficult to get code upstream and released to distributions as it is increasingly unlikely our use-case is interesting to upstream. There were experimental patches to ADB a couple years ago to support flatpak so we didn’t take on this effort without considering our options. Ultimately, this prototype was to see the feasibility of making something that solves our problems while not locking us out of supporting other systems in the future.

Builder happenings for January

I’ve been very busy with Builder since returning from the holidays. As mentioned previously, we’ve moved to gitlab. I’m very happy about it. I can see how this is going to improve the engagement and communication between our existing community and help us keep new contributors.

I made two releases of Builder so far this month. That included both a new stable build (which flatpak users are already using) and a new snapshot for those on developer operating systems like Fedora Rawhide.

The vast majority of my work this month has been on stabilization efforts. Builder is already a very large project. Every moving part we add makes this Rube Goldberg machine just a bit more difficult to maintain. I’ve tried to focus my time on things that are brittle and either improve or replace the designs. I’ve also fixed a good number of memory leaks and safety issues. However, the memory overhead of clang sort of casts a large shadow on all that work. We really need to get clang out of process one of these days.

Over the past couple years, our coding style evolved thanks to new features like g_autoptr() and friends. Every time I come across old-style code during my bug hunts, I clean it up.

Builder learned how to automatically install Flatpak SDK Extensions. These can save you a bunch of time when building your application if you have a complex stack. Things like Rust and Mono can just be pulled in and copied into your app rather than compiled from source on your machine. In doing so, every app that uses the technology can share objects in the OSTree repository, saving disk space and network transfer.

That allowed me to create a new template, a GNOME C♯ template. It uses the Mono SDK extension and gtk-sharp for 3.x. If you want to help here, work on a omni-sharp language server plugin for us!

A new C++ template using Gtkmm was added. Given that I don’t have a lot of recent experience with Gtkmm, it’d be nice to have someone from that community come in and make sure things are in good shape.

I also did some cleanup on our code-indexer to avoid threading in our API. Creating plugins on threads turned out to be rather disastrous, so now we try extra hard to keep things on the main thread with the typical async/finish function pairs.

I created a new messages panel to elevate warnings to the user without them having to run Builder from a terminal. If you want an easy project to work on, we need to go find interesting calls to g_warning() and use ide_context_warning() instead.

Our flatpak plugin now tries extra hard to avoid downloads. Those were really annoying for people when opening builder. It took some troubleshooting in flatpak-builder, and that is fixed now.

In the process of fixing the extraneous downloading I realized we could start bundling flatpak-builder with Builder. After a couple of fixes to flatpak-builder Builder Nightly no longer requires flatpak-builder on the host. That’s one less thing to go wrong for people going through the newcomers work-flow.

We just landed the beginning of a go-langserver plugin. It seems like the language server for Go is pretty new though. We only have a symbol resolver thus far.

I found a fun bug in Vala that caused const gchar * const * parameters to async functions to turn into gchar **, int. It was promptly fixed upstream for us (thanks Rico).

Some 350 commits have landed this month so far, most of them around stabilizing Builder. It’s a good time to start playing with the Nightly branch if you’re into that.

Oh, and after some 33 years on Earth, I finally needed glasses. So I look educated now.


I’ve been quiet since I got back from GUADEC. It’s been a busy summer, but I’ve managed to sneak away and build this in-between my other maintainer/GSoC duties.

There is still plenty to do, but this gets the basic plumbing in place for a debugger.

It can also debug flatpak-based applications (although you’ll need .Debug runtimes for good symbols in gtk/glib/etc).

Under the hood: Runtimes

Now that 3.24 is almost here, I want to take a little downtime and write some information about pieces of the design underneath Builder. Today, let’s lift the hood and take a look at Runtimes.

In Builder, an IdeRuntime is an abstraction around an environment to run programs. That could be build tooling, SDK management, your application, unit tests, and more. Unsurprisingly, it’s naming is similar to Flatpak runtimes because they have a very 1:1 relationship in the Flatpak plugin. But it’s not an uncommon term by any means.

Today, Builder supports 3 types of runtimes: host, jhbuild, and flatpak.

The host runtime simply executes processes on the host system. This is as close to if you just opened a shell with gnome-terminal as we can give you. We actually go through great pains to do so when the application is distributed via Flatpak. For example, we use the HostCommand² developer-mode interface of Flatpak to execute the subprocess from the host session¹ yet using a PTY master/slave from within the sandbox.

The jhbuild runtime is similar to the host runtime except that we first enter the jhbuild environment using jhbuild run. This ensures that we are using dependencies that you may have built locally. Psuedo terminals are also run this way, so ctrl+alt+shift+t will naturally give you the equivalent of jhbuild shell inside a new embedded terminal.

The flatpak runtime uses flatpak build to enter the runtime environment. Our build pipeline will have had to advance to the point at least the IDE_BUILD_PHASE_PREPARE phase to ensure that the manifest and build directories are setup. So there is just a bit extra complexity involved here. But the result is pretty nice. Your subprocess will see /usr, /app and such just like during the application build process. This makes exploring the build environment very simple. Just open a new terminal with ctrl+alt+shift+t.

Running your application naturally uses runtimes too. This allows us to setup the environment, DBus, network namespaces, Wayland access, and more in the way your application needs. We use information from the build configuration (such as your org.gnome.Foo.json flatpak manifest) to determine what that is.

So what’s next?

I’m relatively happy with this design so far and I think it could work for a few more scenarios. One might be a runtime accessible via adb to your phone. Or possibly a remote container. Or far more interesting to me, inside a GNOME virtual machine providing a simulator with GNOME continuous.

Until next time…

  1. Obviously this means Builder has a more relaxed sandboxing story than a general purpose application. This comes with the territory of developer tooling of this magnitude.
  2. If you want to see this feature land in MonoDevelop/Xamarin Studio, go help fix

Simplifying newcomer setup

One thing we are striving for in 3.24 is to make it as simple as possible for newcomers to get their development environment setup. Hopefully in time so that our next round of Outreachy and GSoC interns have an easier time getting started.

A common installation issue we’ve seen is that people have flatpak, but not flatpak-builder. Without it, Builder can’t do builds inside of the target mount namespace with all your proper dependencies. So now Builder will detect this and install it for you if you like.

Sharing your app with friends

It’s a rainy afternoon in Portland so I’m cozy with an espresso watching the rain. After a short hacking session you can now export your application as a Flatpak bundle quickly and easily. Just select the workbench menu in the top right corner of the workbench, followed by Flatpak, and then Export as Bundle.

A normal build will proceed but also includes a finalization step to populate the OSTree repository (flatpak build-export) and then create the bundle (flatpak build-bundle). Finally, Files will be opened with the bundle selected for your copy-and-paste ease. If you double-click the bundle, GNOME Software will be opened and allow you to quickly install the bundle.

Additionally, we’ve split off the update dependencies phase of the build pipeline for Flatpak-based configurations so you can work offline while still updating dependencies at your convenience. Just activate the menu item and the build pipeline will progress to download updates if the network is enabled.

Exploring your application runtime

In Builder, we landed a new feature for 3.24 that allows you to create a new terminal inside the application runtime. If you’re building against your host system, then this is nothing special. If you’re building against jhbuild you’ll get a shell inside of that (but again, nothing really special).

However, if you’re building for a flatpak runtime, such as org.gnome.Platform, your shell will look completely different from your host system. Instead, it will be what the application sees at runtime. (Okay, it’s actually what it sees at build time, but the SDK is just the Platform with compiler bits, headers, pkg-config, and such).

  • ctrl+shift+t for a terminal on your host
  • ctrl+alt+shift+t for a terminal inside your application runtime

So here is an image of a shell in the host system on left and the runtime on the right. This makes it very convenient to figure out what your application is seeing during execution and audit the files you need to remove in your cleanup phase. Or maybe you just want to explore!

If you are not using Builder, you can explore a SDK or runtime with flatpak run org.gnome.Sdk. Additionally, if you’re interested in exactly what your application sees after you have packaged and installed it, try:

flatpak run --command=bash com.example.MyApp

Adding -d to the above will give you the SDK instead of the runtime. This is much closer to what Builder will give you.

flatpak run -d --command=bash com.example.MyApp

Flatpak at SCaLE 15x

A decade ago I lived on the west side of Los Angeles. One of my favorite conferences was Southern California Linux Expo. Much like Karen, this is the conference where I performed my first technical talk. It’s also where I met and became friends with great people like Jono, Ted, Jeff, the fantastic organizing staff, and so many more.

I was happy to come back again this year and talk about what I’ve been working on in GNOME. The combination of Flatpak and Builder (and Sysprof).

The event was live streamed on youtube, and you can watch it here. I expect the rooms to be cut and uploaded as individual talks but I don’t know the timeline for that. I’ll update this if/when I discover that so you can youtube-dl if you prefer.

State of the Builder

We are really starting to be able to do cool things with Builder these days. Just install our Builder Nightly and you can build plenty of GNOME apps.

I should note that Builder doesn’t yet have the ability to install your application but I intend for this to land before 3.24.

Here is a screenshot sequence of me using Builder installed from flatpak to build and test GNOME ToDo, also in flatpak.