Month: October 2016

Builder Rust

With Federico’s wonderful post on Rust’ifying librsvg I guess it makes sense to share what I’ve been doing the last couple of days.

I’ve been keeping my eye on Rust for quite a while. However, I’ve been so heads down with Builder the last two years that I haven’t really gotten to write any or help on integration into our platform. Rust appears to take a very pragmatic stance on integration with systems code (which is primarily C). The C calling convention is not going anywhere, so at some point, you will be integrating with some part of a system that is “C-like”. Allowing us to piecemeal upgrade the “Safety” of our systems is much smarter than rewrite-the-universe. This pragmatism is likely due to the realities of Rust’s birth at Mozilla. It’s a huge code-base, and incrementally modernizing it is the only reality that is approachable.

We too have a lot of code. And like many other projects, we care about being language agnostic to a large degree. In the early days we might have chosen C because it was the only toolchain that worked reliably on Linux (remember C++ pre-2000? or LinuxThreads?) but today we still care about “Language Interopability” and C is the undeniable common denominator.

One way in which we can allow the interopability that we desire and the Safety we need is to start approaching some of our problems like Federico has done. The C calling convention is “Safe”. That is not where zero-day bugs come from. They come from hastily written C code. It is perfectly fine to write our interfaces in C (which can be our basis for GObject Introspection) but implement Safety critical portions in Rust.

This is exactly what I’d like to see from the hundreds of thousands of lines of C I’ve written over the years. We need to identify and fix the Satefy Critical portions. GStreamer is already on the right track here by looking at codec and plugin implementations in Rust. I’m not sure how far they will go with adapting to Rust, but it will be one of the best case-studies we will have to learn from.

So because of this desire to look at building a Safer Platform for developers and users alike, I’ve decided to start adding Rust support to Builder. Thanks to the hard work of the Rust team, it’s a fairly easy project from our side. There is a new Language Server Protocol 2.0 that was worked on by various people at Microsoft, Red Hat, and elsewhere. The new rustls was announced last week and uses this protocol. So I implemented support for both as quick as I could, and now we have something to play with.

Because of the Language Server Protocol, our Rust plugin is tiny. It is essentially a glorified supervisor to handle subprocess crashes and some binding code to connect our stdin/stdout streams to the Language Server Protocol client in Builder. See for yourself.

There is a bunch more work for us in Builder to make it a great Rust IDE. But if people start playing with the language and are willing to work on Builder and GNOME to improve things as a community, we can build a Modern, Safe, and Elegant developer platform.

The big ticket next steps for those that want to contribute to Rust support would include:

  • Cargo build system support. I do believe that ebassi started on something here. But I need to circle back around and sync up.
  • Symbol Tree needs improvements.
  • Semantic highlighter (which we can implement using fuzzy symbols from the symbol tree until a real protocol comes along).
  • Upstream rustls needs work too to get us the features we want. So Rustaceans might want to spend some time helping out upstream.
  • We need to simplify the glue code from Rust←→GObject so that it is dead simple to wrap Rust code in a GObject-based library (where we get our free language interopability).
  • … and of course all the other Builder plumbing that needs to happen in general. See our list of projects that need work.
This image depicts control+clicking on a symbol to jump to its definition. +period or :gd in Vim mode also work.
This image depicts control+clicking on a symbol to jump to its definition. <alt>+period or :gd in Vim mode also work.
This image shows diagnostics displayed over source code.
This image shows diagnostics displayed over source code.
This image shows completion of fields from a struct.
This image shows completion of fields from a struct.
This image shows the Symbol Tree on the right containing elements from the Rust document.
This image shows the Symbol Tree on the right containing elements from the Rust document.

JCON – Part 2

I just finished the JCON_EXTRACT() support and it is already making my consumption of JSON easier. Here is an example:

g_autoptr(JsonNode) node = NULL;
JsonArray *ar = NULL;
gboolean success;

node = JCON_NEW (
  "foo", "{",
    "bar", "[", JCON_INT (1), JCON_INT (2), "]",
  "}"
);

success = JCON_EXTRACT (node,
  "foo", "{",
    "bar", JCONE_ARRAY (ar),
  "}"
);

And for now, you can just copy/paste the jcon.c and jcon.h files into your project, but I’d expect to come up with a patch we can push into json-glib at some point. It really belongs there.

JCON

Years ago during my tenure at MongoDB I worked on a couple libraries. Notably the libbson and mongo-c-driver libraries. One neat feature we had was this concept called BCON which stood for BSON C Object Notation. It was a succinct format for creating BSON documents that made it easier to reason about what you were creating.

So I decided to do the same thing this evening around json-glib because I found I was writing a lot of code to create objects/arrays/etc.

It looks something like this

g_autoptr(JsonNode) params = NULL;

params = JCON_NEW (                                                      
  "changes", "["                                                         
    "{",                                                                 
      "uri", JCON_STRING (src_uri),                                      
      "type", JCON_INT (FILE_CHANGE_TYPE_DELETED),                       
    "}",                                                                 
    "{",                                                                 
      "uri", JCON_STRING (dst_uri),                                      
      "type", JCON_INT (FILE_CHANGE_TYPE_CREATED),                       
    "}",                                                                 
  "]"                                                                    
);

Since this uses va_list it’s technically less type safe than your other options. But it uses some magic struct initializers to get things in a situation where we can bail at runtime if you did something wrong.

The other half that I’m currently still missing is the extraction support. If you replace JCON_NEW() with JCON_EXTRACT(node) and the values with pointers to values, you can quickly extract documents.

But that’s not done yet… so for next time…

jcon.h jcon.c

Sysprof Plans for 3.24

The 3.24 cycle is just getting started, and I have a few plans for Sysprof to give us a more polished profiling experience in Builder. The details can be found on the mailing list.

In particular, I’d love to land support for visualizers. I expect this to happen soon, since there is just a little bit more to work through to make that viable. This will enable us to get a more holistic view of performance and allow us to drill into callgraphs during a certain problematic period of the profile.

Once we have visualizer support, we can start doing really cool things like extracting GPU counters, gdk/clutter frame-clock timing, dbus/cpu/network monitors and whatever else you come up with.

A CPU visualizer displayed above the callgraph to provide additional context to the profiled execution.
A CPU visualizer displayed above the callgraph to provide additional context to the profiled execution.

Additionally we have some work to do around getting access to symbols when we are running in binary stripped environments. This means we can upload a stripped binary to your IoT/low-power device to profile, but have the instruction-pointer-to-symbol resolver happen on the developers workstation.

As I just alluded to, I’d love to see remote profiling happen too. There is some plumbing that needs to occur here, but in general it shouldn’t be terribly complicated.