So this happened…
Everything is still sort of a work-in-progress, but soon I expect to have patches for all the right people in all the right places. There is a lot of slight-of-hand going on here, so it’s worth taking some time to get the details documented.
One of the tools we often find ourselves using when working on GTK+ is sysprof. While doing some profiling recently, I got annoyed that it was one of the few programs on my system that required GTK+ 2.x.
I decided to improve things over the last week and see if we can make something that could be used with Builder too.
First, I needed a library so that we can consume these bits from Builder. So now there is a libsysprof-2 library. The modules have been cleaned up to fit into typical GObject-style library usage.
There were some rough edges on the UI as well. For example, if you tried to launch a profile from the UI, it would fail because non-root users cannot call the __NR_perf_event_open syscall. Getting around this requires running the UI as root (non-ideal) or having a helper process that can run as root and perform the syscall for you. Obviously, I went with the latter. That means there is now a sysprof daemon that uses polkit to check for authorization. GNOME Shell users will see the typical authorization dialog if your DBus connection to the daemon lacks the required credentials.
Additionally, I abstracted the event sources. This will allow us to plug in additional data in the future like cpu, memory, network, systemtap probes, and more.
Due to abstracting the new data sources, I needed a new capture format. I decided to put together a very simple binary capture format for now. It makes things rather simple so we can have mmap() windows as we capture events to disk. We also support memory captures thanks to memfd (and the same mmap window code).
For the inevitable question of “Why not just use the perf capture format?”, my answer right now is simply that I hope to support more than just perf captures and I’d like to have everything in a single file.
I’m going to start merging this into sysprof on git.gnome.org over the next week, but if you want to take a look at things now, you can find the code here.
I really hope that moving this project forward inspires some of you to jump into the code base and help. There is so much we can do in this space. Having great tools helps us all write greater software faster.
If you find yourself with a distcc setup, you might be wondering how to set it up in Builder. Let us assume you’ve setup a distccd on a Fedora 23 server doing something like the following.
# dnf install distcc-server # echo 'OPTIONS="--jobs 24 --allow 192.168.122.0/24"' >> /etc/sysconfig/distccd # systemctl enable distccd.service # service distccd start
Now open your project in Builder and go to the configuration perspective.
Duplicate the configuration and add the CC and DISTCC_HOSTS environment variables like below. I like using -std=gnu11, but you need not do the same.
Keep in mind, that distcc is not magic pixie dust. It isn’t always faster than compiling locally.
Now that we have a basic distcc setup, lets extend it to use ccache to cache compilation results that are exactly the same as previous builds. First make sure you have ccache installed.
# dnf install ccache
Next tweak the environment variables to use ccache gcc as our CC, and instruct ccache to use distcc by setting the CCACHE_PREFIX environment variable to "distcc".
Erick asked recently about what you can extend in Builder. I figured that would be better as a blog post, so hopefully he doesn’t mind the public attention!
Of course, everything in Builder is under development, and there are lots of things that are not yet ready for prime time as plugins. But we are getting there pretty quickly.
Also note, that this list is based on 3.19.90, which was just released last week. It looks a little something like this.
IdeCompletionProvider, which is really just an extension of GtkSourceCompletionProvider allows integration with auto-completion in Builder. There are a bunch of examples in-tree. Clang is rather complex, but there is also Python Jedi, and Ctags.
As discussed previously, IdeApplicationAddin can be thought of singletons within the application. A nice way to share information in your plugin that is not project specific.
An IdeApplicationTool allows creating a sub-command that is accessible using the ide command. Compare this to how you use the git command line tools.
An IdeService is similar to an application addin, but is per-project. So each loaded project in Builder will have the service instantiated. We use this in the clang plugin to manage background AST generation. We then use these ASTs to do things like auto-completion, symbol resolving and symbol tree, semantic highlighting, and more.
When performing builds with Builder, you might want to extract information from the build output. This is exactly what the GCC plugin does to extract build time diagnostics. Your extension will be loaded when building and can receive callbacks or signals during the build process. See IdeBuildResultAddin.
The Build System support in Builder is provided by plugins. Currently, we only support autotools out of the box. However, I’m looking forward to contributions for more build systems. To implement a build system, you’ll need to take a look at IdeBuildSystem, IdeBuildResult, and IdeBuilder.
It’s no secret that my goal for Builder is to raise the bar in terms of developing with external devices. We don’t have much using this yet, but the goal long term is to listen to udev events and show devices as they are added in Builder. See IdeDeviceProvider for adding support here. Long term, this will be integral when interacting with IdeRuntime so that we can setup cross-compilers.
You can even imagine the devices being remote and connected via SSH.
Builder has support to make writing semantic highlighters easier. One of the tough parts of writing a highlighter is doing so at such a rate that does not block the main loop or make the UI stutter. IdeHighlighter is how we do this. Some examples are our clang highlighter and xml highlighter.
While I have grand ideas for new ways to write auto-indenters, our interface IdeIndenter will probably not need to change. This type of thing is always a lot of spaghetti code, rife with backtracking. But in 3.22 I hope to simplify this with a new “selector-based” indentation engine. More on that in the coming months. Currently we have a basic GNU C89 indenter, Python, XML, and Vala.
It’s no secret that I’m not happy with the current sidebar in Builder, but it’s the first step in the direction we want to go. Gotta start somewhere! But the piece that is very useful, is the concept of perspectives. Long term I expect perspectives to be how we avoid the the “jam everything into toolbars” IDE trap. Simply add the IdePerspective to the IdeWorkbench to add a new perspective. We current have 3, the editor, build configurations, and preferences. I expect a couple more by 3.22.
The IdeWorkbenchAddin allows you to tweak the UI of the workbench. Your addin is created once the workbench has loaded a project. You might add a GAction to be used in your UI, add a perspective, or surprise me.
The IdePreferencesAddin is how you add new preferences to Builder. It has a declarative interface to add switches and knobs and be able to remove them when your plugin unloads. This feature is how we avoid the “Configure Plugin” trap I complain so much about in “plugin orientated architectures”.
Unfortunately, last I tried tracker to do project discovery I ended up with about 20 GB of resident memory and then my system crashed. So we currently do project mining manually from Builder. I really hope this isn’t always the case, but it’s where we are today.
Use IdeProjectMiner to implement a project miner and it will get shown when Builder shows the project greeter. We currently only discover autotools projects on your system.
An IdeRuntime is how we manage executing processes within a context such as a jail, container, or host system. Think of it as a toolchain that can be separate from your host system and used for compilation and execution tasks.
The IdeRuntimeProvider interface is how you discover and create these runtimes.
Currently, we have two (well three) implementations. One for Xdg-App runtimes, and one for Jhbuild runtimes. The third is the host system, but that is more of a “no-op” implementation.
I can see more of these being added in the future. Python’s virtualenv, multiple ruby installations, Mono version runtimes, Docker, etc.
When integrating these with an IdeDevice, I hope that we can even get off-system building working inside of virtual-machines (Boxes for example). For me, that would simplify the “Build on Fedora”, “Deploy on RHEL” story.
The search box at the top of Builder is powered by IdeSearchProviders. I really hope to make this feature more powerful in upcoming releases.
The IdeSymbolResolver interface allows you to locate a symbol given a position in a text buffer. This is how we jump to source locations when you type <alt>. in a buffer (or gd in vim mode).
The symbol resolver also helps us create a hierarchy of symbols as found in the Symbol Tree panel.
See the clang and vala implementations.
Version control in Builder is implemented using the IdeVcs interface. Currently, we only support git, thanks to the wonderful libgit2-glib library.
If the version control interface implements the IdeVcs::get_buffer_change_monitor() vfunc, then you will magically get those pretty change lines in the text editor gutter.
If you want to add extensions to every editor created in Builder, use the IdeEditorViewAddin interface.
This interface is rather incomplete today, so I’m really interested in the things you’d like to do so we can make this as easy as possible.
The project greeter, as seen when Builder launches, has support to create new projects. Although, today we only support cloning from Git or from a local directory. In 3.22 we need to extend this to support our new project template support.
To add a new project creation method, implement the IdeGenesisAddin. You can find the git and directory implementations for inspiration.
The basic plumbing for project templates has landed, but it is only exposed via the ide create-project command line program for 3.20. As I said above, I hope to complete this for 3.22 (or mentor someone to complete it for me, hint hint).
Implement the IdeTemplateProvider and IdeProjectTemplate interfaces for how to go about doing this.
The autotools+shared-library, which is written in Python, might serve as some inspiration.
To keep the Builder process lightweight, we advise doing memory and CPU intensive operations in sub-processes. The IdeWorker interface makes this easy. Builder will spawn a worker process for you, and setup a private D-Bus (no daemon required) to communicate to your sub-process.
Longer term, we hope to add more containment features, CPU and memory throttling (with cgroups), and process recycling when memory fragmentation gets absurd.
This would be a great GSoC project if any students are still reading this far down 😉
We have a lot more things to add in the near future. See my work-in-progress ideas for 3.22 features on our Roadmap.
It’s no secret that one of the main features I wanted to land this cycle was introductory support for Xdg-App. There really was quite a bit to do to make that happen, including all sorts of seemingly unrelated plumbing.
One seemingly unrelated piece is the long-anticipated support for “Build Configurations”. I deferred on this feature for as long as possible because it needs to support many movable parts, for which we didn’t have a clear vision of. But now that our applications vision is becoming less and less murky, it was time to attack it.
IdeContext now contains an IdeConfigurationManager. This object can be used to get access to available configurations (IdeConfiguration). A configuration currently contains a target device (IdeDevice), a target runtime (IdeRuntime), environment variables (IdeEnvironment), and various other common properties.
We’ve ambitiously tried to avoid writing random dot files to your project, but time has come to add one. If you create a build configuration, we’ll drop a small .buildconfig GKeyFile in the root of your project tree. It contains the necessary information to bootstrap your project or build it with a given Xdg-App runtime.
Currently, we only have two runtime implementations. The default, is simply the host system. Think of it as a pass-through runtime. The second, is an Xdg-App runtime. It uses the runtime SDK installed via the xdg-app command line tool. For 3.22, I expect to have UI to install runtime SDKs.
You might imagine additional runtimes that we could create. Jhbuild is an obvious one, although hopefully it is less and less necessary now that we have a nightly GNOME SDK.
The widget behind the new preferences implementation in Builder was pretty fun to write. Some of the details were tricky, so I thought I’d make the widget reusable in case others would like to use it. I’m sure you can find uses for something like this.
The widget allocates space for children based on their priority. However, instead of simply growing in one direction, we allocate vertically, but spill over into an adjacent column as necessary. This gives the feeling of something akin to GtkFloxBox, but without the rigidity of aligned rows and columns.
This widget is meant to be used within a GtkScrolledWindow (well GtkViewport really) where the GtkScrolledWindow:hscrollbar-policy is set to GTK_POLICY_NEVER.
The resulting work is called EggColumnLayout and you can find it with all the other Egg hacks.
A short description of the tune-ables are:
EggColumnLayout:column-width tunes the width of the columns. These are uniform among all columns so you should set it to something reasonable. The default is 500px. As an aside, dynamically descovering the column width uniform to all children would probably not look great, nor be straight-forward.
EggColumnLayout:column-spacing is the spacing between columns. Had we used children containers, we could have probably done this in CSS, but the property is “good enough” in my opinion.
EggColumnLayout:row-spacing is the analog to :column-spacing but for the space between the bottom of one child and the top of a subsequent child.
That is pretty much it. Just add your children to this and put the whole thing in a GtkScrolledWindow.
Happy Hacking.
Now that Builder has integrated Template-GLib, I started working on creating projects from templates. Today, this is only supported from the command line. Obviously the goal is to have it in the UI side-by-side with other project creation methods.
I’ve put together a demo for creating a new shared-library project with autotools. Once I’m happy with the design, I’ll document project templates so others can easily contribute new templates.
Anyway, you can give it a go as follows.
ide create-project my-project -t shared-library cd my-project ide build
This is a friendly reminder that we have a Developer Experience Hackfest starting in a few days as the warm-up to the, ever so energizing, FOSDEM. Come help us build cool things for people who build cool things!
I will be working on all things Builder related. It’s your chance to trick me into writing your pet feature.
I’m doing some research on panel systems that people love, hate, and love to hate. Send me a short email with the following to get your voice heard. No promises of implementation, but I’d like to inform myself nonetheless.
Name: Application or Library Name
Screenshots: 1-2 screenshot links
Likes: 1-2 Sentence max of likes
Dislikes: 1-2 Sentence max of dislikes
Previously, Builder Plugins, Part I. Read this if you haven’t.
Now that we’ve learned the mechanics of creating a plugin, let’s look at another plugin interface, IdeWorkbenchAddin. This interface allows us to extend the IdeWorkbench. An IdeWorkbench is the toplevel window for a given project. Since Builder supports having more than one project open at a time (each in a given workbench window), this is the interface you would use for extensions that are per-project.
The workbench window has an important property. The IdeWorkbench:context. This is the IdeContext. Each loaded project has an IdeContext. This is the primary data structure that provides access to your Version Control System, Build System, Device Manager, and many other components.
The two important virtual functions in the IdeWorkbenchAddin interface are IdeWorkbenchAddin::load and IdeWorkbenchAddin::unload. These provide an IdeWorkbench as a parameter, for which you can use to your hearts content. IdeWorkbenchAddin::unload() should reverse anything that happened in IdeWorkbenchAddin::load().
The other virtual functions are optional, and can be used to allow your IdeWorkbenchAddin to perform custom actions upon opening URIs and such. I’m sure we’ll add more as we go.
Anyway, let’s create a sample IdeWorkbenchAddin that prints some information about our loaded project.
# my_plugin.py
from gi.repository import Ide
from gi.repository import GObject
class MyWorkbenchAddin(GObject.Object, Ide.WorkbenchAddin):
def do_load(self, workbench):
self.workbench = workbench
context = workbench.props.context
vcs = context.props.vcs
build_system = context.props.build_system
print("Version Control System is:", repr(vcs))
print("Build System is:", repr(build_system))
def do_unload(self, workbench):
self.workbench = None
Use the same my_plugin.plugin found in Part I.
Running Builder should result in something like this on the command line.
Version Control System is: <__gi__.IdeGitVcs object at 0x7fffbd8de438 (IdeGitVcs at 0x1caaee0)> Build System is: <__gi__.IdeAutotoolsBuildSystem object at 0x7fffbd8de798 (IdeAutotoolsBuildSystem at 0x1d00890)>
You can see that my project uses Git for Version Control, and Autotools for the build system. Nothing too spectacular here, so now we’ll move on to something a bit more complex.
See each of the icons on the left of the workbench? Those are what we call perspectives. Currently, we only have two perspectives. One for the editor, and one for preferences. I’m sure you can imagine a bunch of new perspectives. Bugzilla, git, debugger, profiler, database browser, and designer all come to mind.
I should mention that I’m not sure whether or not we’ll keep the perspective bar like this. It’s been there since some of the earliest designs, but we may opt for something slightly different.
Like many of the other extension points in Builder, IdePerspective is an interface you can implement in your library. However, you need to add it to the workbench using IdeWorkbench.add_perspective(). As always, you should remove it when you unload your plugin, using IdeWorkbench.remove_perspective().
Lets create a Hello World perspective.
# my_plugin.py
from gi.repository import Ide
from gi.repository import GObject
from gi.repository import Gtk
class MyWorkbenchAddin(GObject.Object, Ide.WorkbenchAddin):
def do_load(self, workbench):
self.perspective = HelloPerspective(visible=True)
workbench.add_perspective(self.perspective)
def do_unload(self, workbench):
workbench.remove_perspective(self.perspective)
self.perspective = None
class HelloPerspective(Gtk.Box, Ide.Perspective):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
label = Gtk.Label(label="Hello, World!",
expand=True,
visible=True)
self.add(label)
self.titlebar = Ide.WorkbenchHeaderBar(visible=True)
def do_get_id(self):
return 'hello-world'
def do_get_title(self):
return 'Hello'
def do_get_priority(self):
return 10000
def do_get_icon_name(self):
return 'image-missing'
def do_get_titlebar(self):
return self.titlebar
Lets run Builder and see what happens!
More next time, same bat channel.