Talk went well today, you can grab a copy of the slides here.
Talk went well today, you can grab a copy of the slides here.
A reminder that I’ll be speaking at Scale 17x in Pasadena on Sunday, March 10th about all the cool stuff we’ve been doing in Builder and how that plays into the role of modernizing our development stack.
Also, Sri is speaking. Matthias too.
As I mentioned in my overview of the upcoming Builder release, a lot of this development cycle focused on improving machinery the user does not see. In the past four years of development, a number of patterns emerged. When I started this project, I had some ideas of how to keep things organized, but Builder quickly grew beyond my ability to keep the whole design in my head at once.
Thankfully, I’ve worked on many large software teams during my tenure at companies like VMware and MongoDB. These company’s products have something in common, in that they’re a number of sub-systems performing specialized tasks that coordinate with each other. Not surprising, I know. But sometimes, knowing where to draw the lines between sub-systems is what differentiates products. (Interestingly, if you want to build a database engine, you better have a good reason to deviate from page 144 of Architecture of a Database System).
Now days, Builder segments the code into a series of static libraries. We have a core library that provides the basic functionality used by all sub-systems. We have a process and threading library. One library deals with representing and manipulating source code. Yet another focuses on building, executing, testing, and packaging software. To enable Language Servers, we have one too. Even the gui and editor are somewhat compartmentalized.
In the end, we link all of these static libraries into the final executable, which is then processed to generate GObject Introspection data; the *.gir
and *.typelib
. That means that our plugins, written in C, C++, Python, or Vala, do not need to rely on linking against any shared libraries. All symbols come from the executable with -export-dynamic
. (One side-effect of this, is that we no longer allow linking our Vala based plugin into the executable, since it relies on *.gir
generation).
To keep startup fast, all of our bundled C-based plugins are compiled into the final executable. That reduces the number of link loader work, file-system directory scanning and iops, and allows for more aggressive compiler optimizations. We also avoid loading some plugins in the early initialization phase, instead deferring that work until later.
The biggest change in the refactoring is IdeObject. This object has become somewhat like a multi-threaded version of what GtkObject
used to be. It allows us to build a top-down object tree. The root of the tree is a specialized form, IdeContext. Each sub-system mounts itself on this tree. Plugins may have or work with objects that are descendants of the sub-systems. In the end, it makes things much easier for me to debug at runtime because I can get a clearer picture of how things interact and what plugins are active. One interesting feature of IdeObject
is that we guarantee they are only ever finalized in the main thread.
The sub-system that I think resulted in the most cleanup was libide-foundry.a
. This contains the build manager, pipelines, and extension points for plugins to provide the the abstraction glue. It also contains runtime abstractions, execution management, testing infrastructure, device management (phone, tablet, simulator), and toolchains. Currently, the debugger engine is a separate library, but I may fold it into this library for completeness.
When cleaning up libide-code.a
, I simplified the buffer management. This code was a bit temperamental because I put a few too many features into the base class. I think that happened simply because I couldn’t yet foresee many of the ways it would be extended. Seeing how the code evolved over time allowed me to create better abstractions and extract those features.
What kicked off this whole process was an hour hack to add multi-monitor support. That somehow turned into a 2 month project that involved me ripping everything apart and putting it back together because of our initial design. I’m happy to say that I think things turned out quite well and we are poised to keep moving fast for the next few development cycles to come.
We’re getting many more requests to support external plugins in Builder. I’ve always taken the Linux kernel approach here in that I very much want to encourage upstream contributions over external plugins. However, I am starting to loosen up just a bit and we’ve added the X-Builder-ABI=3.31
key to plugin definitions. This allows us to continue to evolve our APIs but promise to not break them in micro releases. My expectation, however, is that this major refactoring will set us down the path towards some amount of ABI stability.
In my next post I’d like to cover how the UI components are abstracted and give you an idea of which plugin interfaces to use and when. In the mean time, I have plenty of documentation to write.
We just landed the largest refactor to Builder since it’s inception. Somewhere around 100,000 lines of code where touched which is substantial for a single development cycle. I wrote a few tools to help us do that work, because that’s really the only way to do such a large refactor.
Not only does the refactor make things easier for us to maintain but it will make things easier for contributors to write new plugins. In a future blog post I’ll cover some of the new design that makes it possible.
Let’s take a look at some of the changes in Builder for 3.32 as users will see them.
First we have the greeter. It looks similar as before, although with a design refresh. But from a code standpoint, it no longer shares it’s windowing with the project workspace. Taking this approach allowed us to simplify Builder’s code and allows for a new feature you’ll see later.
Builder now gives some feedback about what files were removed when cleaning up old projects.
Builder gained support for more command-line options which can prove useful in simplifying your applications setup procedure. For example, you can run gnome-builder --clone https://gitlab.gnome.org/GNOME/gnome-builder.git
to be taken directly to the clone dialog for a given URL.
The clone activity provides various messaging in case you need to debug some issues during the transfer. I may hide this behind a revealer by default, I haven’t decided yet.
Creating a new project allows specifying an application-id, which is good form for desktop applications.
We also moved the “Continue” button out of the header bar and placed it alongside content since a number of users had difficulty there.
The “omni-bar” (center of header bar) has gained support for moving through notifications when multiple are present. It can also display buttons and operational progress for rich notifications.
Completion hasn’t changed much since last cycle. Still there, still works.
Notifications that support progress can also be viewed from our progress popover similar to Nautilus and Epiphany. Getting that circle-pause-button aligned correctly was far more troublesome than you’d imagine.
The command-bar has been extracted from the bottom of the screen into a more prominent position. I do expect some iteration on design over the next cycle. I’ve also considered merging it into the global search, but I’m still undecided.
Also on display is the new project-less mode. If you open Builder for a specific file via Nautilus or gnome-builder foo.c
you’ll get this mode. It doesn’t have access to the foundry, however. (The foundry contains build management and other project-based features).
The refactoring not only allowed for project-less mode but also basic multi-monitor support. You can now open a new workspace window and place it on another monitor. This can be helpful for headers, documentation, or other references.
The project tree has support for unit tests and build targets in addition to files.
Build Preferences has been rebuilt to allow plugins to extend the view. That means we’ll be able to add features like toggle buttons for meson_options.txt
or toggling various clang/gcc sanitizers from the Meson plugin.
The debugger has gone through a number of improvements for resilience with modern gdb.
When Builder is full-screen, the header bar slides in more reliably now thanks to a fix I merged in gtk-3-24.
As previewed earlier in the cycle, we have rudimentary glade integration.
Also displayed here, you can select a Build Target from the project tree and run it using a registered IdeRunHandler
.
Files with diagnostics registered can have that information displayed in the project tree.
The document preferences have been simplified and extracted from the side panel.
The terminal now can highlight filename:line:column
patterns and allow you to ctrl+click to open just like URLs.
In a future post, we’ll cover some of what went into the refactoring. I’d like to discuss how the source tree is organized into a series of static libraries and how internal plugins are used to bridge subsystems to avoid layering violations. We also have a number of simplified interfaces for plugin authors and are beginning to have a story around ABI promises to allow for external plugins.
If you just can’t wait, you can play around with it now (and report bugs).
flatpak install https://gitlab.gnome.org/GNOME/gnome-apps-nightly/raw/master/gnome-builder.flatpakref
Until next time, Happy Hacking!
One of the things we’ve wanted in Builder for a while is a designer. We’ve had various prototypes in the past to see how things would have worked out, and mostly just punted on the idea because it seemed like Glade served users better than we would be able to directly.
Last week, Juan Pablo, Matthias Clasen and I met up in San Francisco to see what we could do in the short term. We discussed a couple of options that we have going forward.
Like all projects, we have some constraints.
With that in mind, I want to get the maximal benefit with the least amount of time to ship. I made a new plugin for Builder last week that gets us moving in that direction. It still needs more work to integrate with signal editing, templates, and other more advanced Glade features.
Hopefully that happens soon because I know we’ve all been waiting for it. Get it now with the Builder Nightly flatpak.
flatpak install --from https://git.gnome.org/browse/gnome-apps-nightly/plain/gnome-builder.flatpakref
Years ago, when starting the Builder project, I added a real-time CPU graph so that it was easy to detect slow-downs quickly while hacking on the product. If I ever saw a main-loop stall, it would be immediately obvious.
Over the years I added more advanced features like a Sysprof-based profiler. That duplicated the feature and requires significantly less overhead. You also have the added benefit of zooming and panning.
So in a decision that was long overdue, I’m removing the real-time graph from Builder 3.32. I never did a great job of porting that code to optimal Wayland use anyway. It was really designed with Xrender
/Xshm
in mind where XCopyArea()
was cheap and done on the GPU.
There are better solutions now anyway.
Now that org.gnome.Sdk//3.30 has gone gold, you can get Builder from Flathub.
I expect those that have not been using Nightly for the past few months will be pleasantly surprised at the improvements.
People have asked for more advanced session restore for quite some time, and now Builder can do it. Builder will now restore your previous session, and in particular, horizontal and vertical splits.
Like previously, you can disable session restore in preferences if that’s not your jam.
You can write plugins which hook into session save/restore by implementing IdeSessionAddin.
https://www.youtube.com/watch?v=dtdpUP6Qges&feature=youtu.be
For a subset of GNOME modules, I’m still using jhbuild. I also spend a great deal of time tracking down memory bugs in various libraries. So it is very handy to have libasan.so
working with meson -Db_sanitize=address
.
To make things work, you currently need to:
-Db_sanitize=address
.~/.config/jhbuildrc
.Here is an example of what I put in ~/.config/lsan_suppressions.txt
.
leak:FcCharSetCreate leak:FcLangSetCreate leak:__nptl_deallocate_tsd leak:_g_info_new_full leak:dconf_engine_watch_fast leak:g_get_language_names_with_category leak:g_intern_string leak:g_io_module_new leak:g_quark_init leak:libfontconfig.so.1
And add this to ~/.config/jhbuildrc
.
import os os.environ['LSAN_OPTIONS'] = 'suppressions=' + \ os.path.expanduser('~/.config/lsan_suppressions.txt')
This has helped me track down a number of bugs in various modules this week and it would be useful if other people were doing it too.
One dead give-away of a GNOME/Gtk programmer is how they format their headers. For the better part of two decades, many of us have been trying to keep things aligned. Whether this is cargo-culted or of real benefit depends on the reader. Generally, I find them easier to filter through.
Unfortunately, none of indent
, clang-format
, nor uncrustify
have managed to exactly represent our style which makes automated code formatting tools rather problematic. Especially in an automated fashion.
For example, notice how the types and trailing asterisks, stay aligned, in multiple directions.
FOO_EXPORT void foo_do_something_async (Foo *self, const gchar * const *params, GCancellable *cancellable, GAsyncReadyCallback callback, gpointer user_data); FOO_EXPORT Bar *foo_do_something_finish (Foo *self, GAsyncResult *result, GError **error);
Keeping that sort of code aligned is quite a pain. Even for vim users who can fairly easily repeat commands. Worse, it can explode patches into unreadable messiness.
Anyway, I added a new command in Builder last night that will format these in this style so long as you don’t do anything to trip it up. Just select a block of function declarations, and run format-decls
from the command bar.
It doesn’t yet handle vtable entries, but that shouldn’t be too painful. Also, it doesn’t handle miscellaneous other C code in-between declarations (except G_GNUC_*
macros, __attribute_()
etc.