Meson considerations

A post with my GNOME release team hat on…

Meson is new and cool

A number of GNOME modules are switching to meson for 3.26. I myself was an early adopter for this: recipes has had meson build support since the beginning of the year, and after the 1.0 release, I’ve dropped autotools support on the master branch.

autotools are of course very familiar to most of us, and we know how to get most things done there. But it often isn’t pretty, and  using meson feels like a breath of fresh air. Others have been praising meson for its simplicity, ease of use and speed, so I am not going repeat that here.

Supported tools

jhbuild is our traditional build support tool, and it has well-working meson support for a while now. GNOME builder and flatpak-builder also both support meson and we include meson in the GNOME sdk for 3.24.

So, for developers, meson support is more or less there, and working well.

Transition woes

So, things are pretty awesome all around: we have a new build system, it is shiny and fast and supported. Sounds too good to be true. Whats the catch ?

One thing that meson does not do is building traditional ‘make dist’ style tarballs. The premise is that you can just build your software from a git tag or from a snapshot produced by git-archive.

While that is true, and maybe a direction we want to be going in for the future, there are plenty of build systems out there that expect you to provide a tarball or similar archive. That is true for Fedora’s koji, and it is also true for form in which we currently produce GNOME releases.

A GNOME release is essentially defined by a jhbuild module file (several of them, in fact) which refers to release tarballs for all of our modules, including checksums and sizes.  For core GNOME modules, these tarballs are generally put in place using a tool called ftpadmin. As I’ve recently found out, ftpadmin is a little picky. It expects the content in the tarball to be in a directory that’s named in module-version style and will error out if that is not the case.

Thankfully, git archive is up to the task. Here is what I did to produce a recent gnome-recipes release:

git tag -m 1.2.0 1.2.0
git archive --prefix gnome-recipes-1.2.0/ \
            -o gnome-recipes-1.2.0.tar.xz \
            1.2.0

Some unsolved problems remain. For example, we have not decided on how to handle library documentation in the new meson world. The way this works with autotools is that the tarballs include generated docs, which get extracted and post-processed by some scripts before they end up on developer.gnome.org. But git archive snapshots contain no generated documentation…  So far, no library that we host documentation for has made the jump to meson-only builds, so we still have some time to come up with a different solution.

Overall, I am really excited that we are embracing meson!

Update: My discussion of archives failed to consider git submodules. git-archive does not handle those, so my recommendation will not produce a working snapshot if you use submodules. See nautilus’ make_release.sh script for how to handle that.

GTK+ happenings

This will be a longer post summarizing some of the discussions and decisions at the recent GTK+ hackfest in London.

I’ve just released GTK+ 3.90, as a milestone on our way towards GTK+ 4. This post should explain not just what is or isn’t in that release, but also where we are on the journey towards GTK+ 4, and what changes are still in the pipeline.

The reason for doing a 3.90 release now and not just another 3.89.x milestone is that we want to encourage experimental ports of some GTK+ 3 applications at this point, to gather feedback.

If you want to do this for your application, you should be aware that we are not promising more than 6 month API stability yet. There are still a number of unfinished transitions and fundamental changes (such as dropping subwindows, or moving container APIs to GtkWidget) that can and will affect application APIs before GTK+ 4.

I am willing to do a few 3.90.x snapshots if there is interest, but you should also be prepared to keep your application building against GTK+ 3, since 3.90 may not be widely available in distributions.

Hackfest happenings

We had a 3 productive days in London last week. The hackfest was a bit different from previous GTK+ get-togethers since we had a bigger and more diverse audience, which means that we had less down-in-the-details discussion about GTK+ internals.

Instead, some of us spent time discussion portals, Flatpak, build services and other things – but I will focus on the GTK+ parts in this post.

The GTK+ discussion was sometimes a bit higher-level, with topics like graphics pipelines and how we use them in GSK, or constraints and how they can fit into the GTK+ layout machinery.

I’ve personally enjoyed this change of pace quite a bit, and hope we can repeat this style of event in the future.

The details: GSK

We spent a while reviewing how far we have come with rebasing the GTK+ rendering onto GSK (and thus, GL / Vulkan). I won’t go into excessive details here, but we have both a Vulkan and a GL renderer for GSK.

The Vulkan renderer has shader implementations for many of the performance-relevant elements of CSS that make up GTK+ widgets. One major omission is that we still don’t handle text natively and fall back to cairo rendering and texture upload for it. We decided to look at cogl-pango and lift the necessary texture atlas and pango renderer from there, as far as possible.

The GL renderer is much less complete, since Benjamin has focused on implementing things on the Vulkan side at first. The plan here is to finish the Vulkan side first and then port things over to GL. One difference between the backends that we discussed at some length is that while changing pipelines is cheap with Vulkan, that may not be the case on the GL side, so we may need an optimizing pass over the render node tree to minimize state changes.

The details: constraints

Emmanuele gave a demo of Emeus, which is his implementation of constraints-based layout for GTK+. This demo left me feeling quite positive – everybody seems to be moving in the direction of constraints-based layout.

Emeus also supports Apple’s shorthand notation for constraints, so it should be familiar to many people who don’t have experience with GTK+’s nested boxes model. If we can integrate this in time for GTK+ 4, it could make a real difference for how GTK+ applications are designed and implemented.

In the discussion after the demo, we touched on questions of how we could possibly reimplement existing GTK+ containers in terms of constraints, and how we can take even more advantage of constraints by making Emeus composable.

Of course, there are questions of how far this approach will scale – could you put all of gtk-widget-factory into a single system of constraints and solve it while resizing the window ? We will find out…

The details: input

One of the reasons why we are moving to GL-based rendering is that we want to enable more pervasive animations in many places. One example that we’ve discussed is animating the showing and hiding of list or grid elements in a filtered list box or flow box. Doing this smoothly at 60 frames per second requires that we can move and transform elements without causing full relayouts, which in turn means that we to separate positions from sizes, which is only practical in a world without subwindows.

Therefore, one of the milestones on our journey towards GTK+ 4 is to move event handling away from using subwindows for routing input events to their target. Once we have taken this step, it will also be possible to take transformations into account.

Carlos just posted a branch that is taking the first steps in this direction.

The details: children, nodes and gadgets

I’ve already talked about Emeus and constraints, so I won’t repeat that here. But there are some other changes in the way GTK+ widgets work that are worth mentioning here.

We are moving away from GtkContainer as the parent class for container. Instead we now allow any widget to have children. The APIs for this are more DOM-like than GtkContainer: gtk_widget_get_first_child, gtk_widget_get_last_child, gtk_widget_get_next_sibling, gtk_widget_get_prev_sibling.

In GTK+ 3.20 and afterwards, we introduced gadgets as internal construct that allowed us to clean up our widget’s adherence to the CSS box model. This was always meant as a stop-gap solution for pre-existing widgets that were just too complicated to fit the CSS box model well. Now that any widget can have children, we have begun to replace some of these gadgets with regular widgets.

For example, GtkSwitch now has three child widgets: the slider and two labels.

Summary

We’ve had a successful hackfest and GTK+ 4 development is on track, with 3.90 as a visible milestone. Try it out!

Recipes 1.0

Recipes 1.0 is here, in time for GNOME 3.24 next week. You can get it here:

https://download.gnome.org/sources/gnome-recipes/1.0/

A flatpak is available here:

https://matthiasclasen.github.io/recipes-releases/gnome-recipes.flatpakref

and can be installed with

flatpak install https://matthiasclasen.github.io/recipes-releases/gnome-recipes.flatpakref

Thanks to everybody who helped us to reach this point by contributing recipes, sending patches, translations or bug reports!

Documentation

Recipes looks pretty good in GNOME Software already, but one thing is missing: No documentation costs us a perfect rating. Thankfully, Paul Cutler has shown up and started to fill this gap, so we can get the last icon turned blue with the next release.

Since one of the goals of Recipes is to be an exemplaric Flatpak app, I took this opportunity to investigate how we can handle documentation for sandboxed applications.

One option is to just put all the docs on the web and launch a web browser, but that feels a bit like cheating. Another option is to export all the documentation files from the sandbox and launch the host help browser on it. But this would require us to recursively export an entire directory full of possibly malicious content – so far, we’ve been careful to only export individual, known files like the desktop file or the app icon.

Therefore, we decided that we should instead ship a help browser in the GNOME runtime and launch it inside the sandbox. This turns out to work reasonably well, and will be used by more GNOME apps in the near future.

Interns

Apart from this ongoing work on documentation, a number of bug fixes and small improvements have found there way into the 1.0 release. For example, you can now find recipes by searching for the chef by name. And we ask for confirmation if you are about to close the window with unsaved changes.

Some of these changes were contributed by prospective Outreachy interns.

Roadmap

I have mentioned it before, you can find some information about our future plans for recipes here:

https://wiki.gnome.org/Apps/Recipes/Development

Your help is more than welcome!

A journey, with recipes

This month, we will release GNOME 3.24, and alongside, GNOME recipes will have its 1.0 release.

It has been quite a journey from just an idea at GUADEC last August to the application that we have now.

To the finish line

Since my last update, we have focused on completing our 1.0 goals. The last thing that we needed was to get enough contributed recipes to replace all the test data with actual content. And we’ve made it, thanks to the cooks in the GNOME community, we have plenty of great recipes now:

A few small features have still made their way in, despite the focus on polish. One thing I’ve played with is making appstream data useful inside the application itself, for showing ‘What’s New’ style information:

The timer support in cooking mode has seen quite a bit of improvement and polish.

I’ve also picked up my earlier experiment again, and built Recipes on OS X. This time, I got far enough to produce a dmg image:

A useful by-product of this effort was a number of bug fixes for the GTK+ Quartz backend, such as working fullscreen and window functions.

What next ?

1.0 will not be the end of the road for Recipes. There’s a number of exciting features on the post-1.0 roadmap.

For the next leg of the journey, we will welcome some Outreachy interns who will help us to add useful functionality, from unit conversion to shopping list export.

Recipes by mail

Since I last wrote about GNOME recipes, we’ve mainly focused on completing our feature set for 3.24.

Todays 0.12.0 release brings us very close to covering all the goals we’ve set ourselves when we started working on recipes: We have a  fullscreen cooking mode

and recipes and shopping lists can be shared by email

Since the Share button has replaced the Export button, contributing your recipes is now easier too: Just choose the “Contribute” option and send the recipe to the new recipes-list@gnome.org mailing list.

While working on this, it suddenly dawned on my why I may have seen some recipe contributions in bugzilla that where missing attachments: bugzilla has a limit for the sizes of attachments it allows, and recipes with photos may hit this limit.

So, if you’ve tried to contributed a recipe via bugzilla, and ran into this problem, please send your recipe to recipes-list@gnome.org instead.

Presentation woes

My flatpak presentation at devconf.cz ran into some technical difficulties when the beamer system failed halfway through. I couldn’t show the second half of my slides, and had to improvise a bit. If you were in the room, I hope it wasn’t too incomprehensible.

You can see what you missed here:

And here is a quick recording of the demo I would have given at the end of my talk:

Debugging a Flatpak application

Since I’ve been asking people to try the recipes app with Flatpak, I can’t complain too much if I get bug reports back. But how does one create a useful bug report when something goes wrong in a Flatpak sandbox ? Some of the stacktraces I’ve seen have not been very useful, since they are lacking symbols.

This post is a quick attempt to spread some basics about Flatpak debugging.

Normally, you run your Flatpak app like this:

flatpak run org.gnome.Recipes

Well, that’s not quite true; the ”normal” way to launch the Flatpak is just the same as launching a non-Flatpak app: click on the icon, or hit the Super key, type recipes, hit Enter. But lets assume you’re launching flatpak from the commandline.

What happens behind the scenes here is that flatpak finds the metadata for org.gnome.Recipes, determines which runtime it needs, sets up the sandbox by mounting the app in /app and the runtime in /usr, does some more sandboxy stuff, and eventually launches the app.

First problem for bug reporting: we want to run the app under gdb to get a stacktrace when it crashes.  Here is how you do that:

flatpak run --command=sh org.gnome.Recipes

Running this command, you’ll end up with a shell prompt ”inside” the recipes sandbox.  This is great, because we can now launch our app under gdb (note that the application gets installed in the /app prefix):

$ gdb /app/bin/recipes

Except… this fails because there is no gdb. Remember that we are inside the sandbox, so we can only run what is either shipped with the app in /app/bin or with the runtime in /usr/bin.  And gdb is not among either.

Thankfully, for each runtime, there is a corresponding sdk, which is just like the runtime, except it includes the stuff you need to develop and debug: headers, compilers, debuggers and other useful tools. And flatpak has a handy commandline option to use the sdk instead of the regular runtime:

flatpak run --devel --command=sh org.gnome.Recipes

The –devel option tells flatpak to use the sdk instead of the runtime  and do some other things that make debugging in the sandbox work.

Now for the last trick: I was complaining about stacktraces without symbols at the beginning. In rpm-based distributions, the debug symbols are split off into debuginfo packages. Flatpak does something similar and splits all the debug information of runtimes and apps into separate ”runtime extensions”, which by convention have .Debug appended to their name. So the debug info for org.gnome.Recipes is in the org.gnome.Recipes.Debug extension.

When you use the –devel option, flatpak automatically includes the Debug extensions for the application and runtime, if they are available. So, for the most useful stacktraces, make sure that you have the Debug extensions for the apps and runtimes in question installed.

Hope this helps!

Most of this information was taken from the Flatpak wiki.

Recipes for you and me

Since I’ve last written about recipes, we’ve started to figure out what we can achieve in time for GNOME 3.24, with an eye towards delivering a useful application. The result is this plan, which should be doable.

But: your help is needed. We need more recipe contributions from the GNOME community to have a well-populated initial experience. Everybody who contributes a recipe before 3.24 will get a little thank-you from us, so don’t delay…

The 0.8.0 release that I’ve just created already contains the first steps of this plan. One thing we decided is that we don’t have the time and resources to make the ingredients view useful by March, so the Ingredients tab is gone for now.

At the same time, there’s a new feature here, and that is the blue tile leading to the shopping list view:

The design for this page is still a bit up in the air, so you should expect this to change in the next releases. I decided to merge it already anyway, since I am impatient, and this view already provides useful functionality. You can print the shopping list:

Beyond this, I’ve spent some time on polishing and fixing bugs. One thing that I’ve discovered to my embarrassment earlier this week is that exporting recipes from the flatpak did not actually work. I had only ever tested this with an un-sandboxed local build.

Sorry to everyone who tried to export their recipe and was left wondering why it didn’t work!

We’ve now fixed all the bugs that were involved here, both in recipes and in the file chooser portal and in the portal infrastructure itself, and exporting recipes works fine with the current flatpak, which, as always, you can install from here:

https://alexlarsson.github.io/test-releases/gnome-recipes.flatpakref

One related issue that became apparent during this bug hunt is that things work less than perfectly if the portals are not present on the host system. Until that becomes less likely, I’ve added a bit of code to make the failure less mysterious, and give you some idea how to fix it:

I think recipes is proving its value as  a test bed and early adopter for flatpak and portals. At this point, it is using the file chooser portal, the account information portal, the print portal, the notification portal, the session inhibit portal, and it would also use the sharing portal, if we had that already.

I shouldn’t close this post without mentioning that you will have a chance to hear a bit from Elvin about the genesis of this application in the Fosdem design devroom. See you there!

GTK+ Happenings

I said that I would post regular updates on what is happening in GTK+ 4 land. This was a while ago, so an update is overdue.

So, whats new ?

Cleanup

Deprecation cleanup has continued, and is mostly done at this point. We have the beginning of a porting guide that mentions some of the required changes for early adopters who want to stick their toes into the GTK+ 4 waters. Sadly, I haven’t gotten the GTK+ 4 docs up on the website yet, so no link…

Among the things that have been dropped as part of our ongoing cleanup has been the pixel cache, which should no longer be needed. This is nice since the pixel cache was causing problems, in particular on connection with transparency and component alpha (in font rendering).

Not really a cleanup, but we also got rid of the split into multiple shared objects (libgtk, libgdk, libgsk). Now, we just install a single libgtk, which also provides the gdk and gsk APIs. This has some small performance benefits, but mainly, it makes it easier for us to have private APIs that cross the gtk/gdk boundary.

Widget APIs

Some of the core APIs that are important when you are creating your own widgets have been changed around a bit:

  • The five different virtual functions that are used for size requisition have been replaced by a single new vfunc, measure(). This is using the same approach that we are already using for gadgets, where it has worked well.
  • The draw() virtual function that lets widget render themselves onto a cairo surface has been replaced by the new snapshot() vfunc, which lets widget create render nodes. This is essentially the change from direct to indirect rendering. Most widgets and gadgets have been ported over to this new wayof doing things.

These changes are only important to you if you create your own widgets.

Window APIs

GdkWindow has gained a few new constructors to replace the old libX11-style gdk_window_new.  Their names should indicate what they are good for:

  • gdk_window_new_toplevel
  • gdk_window_new_popup
  • gdk_window_new_temp
  • gtk_window_new_child
  • gdk_window_new_input
  • gdk_wayland_window_new_subsurface
  • gdk_x11_window_foreign_new_for_display

The last two are worth mentioning as examples where we move backend-specific functionality to backend APIs.

In the medium term, we are moving towards a world with only toplevel windows. As a first step towards this, we no longer support native child windows, and gdk_window_reparent() is gone. This allowed us to considerably simply the GdkWindow code.

Renderers

When we initially merged GSK, it had a GL renderer and a software fallback (using cairo). Since then, Benjamin has created a Vulkan renderer. The renderer can be selected using the GSK_RENDERER environment variable.

So, for example, this is how to run gtk4-demo with the cairo renderer and the X11 backend:

GSK_RENDERER=cairo GDK_BACKEND=x11 gtk4-demo

After the GSK merge, we struggled a bit to come up with a working approach to converting all our widget and CSS rendering to render nodes. With the introduction of the snapshot() vfunc, we’ve been able to make progress on this front. As part of this effort, Benjamin changed the GSK API around a bit. There are now a bunch of special-purpose render node subclasses that let us effectively translate the CSS rendering, e.g.

  • gsk_linear_gradient_node_new
  • gsk_texture_node_new
  • gsk_color_node_new
  • gsk_border_node_new
  • gsk_transform_node_new

…and so on. More node types will be created as we discover the need for them.

New fun

As an example of new functionality that would be very hard to support adequately in GTK+ 3, Benjamin recently added gsk_color_matrix_node_new and used it to implement the CSS filter spec, which is good for a few screenshots:

Since this is all done on the GPU (unless you are using the software renderer), applying one of these filters does not affect performance much, as can be seen in this screencast of the fishbox demo:

Expect to see more uses of these new capabilities in GTK+ as things progress. Fun times ahead!

On recipes, one more time

I’m still not quite done with this project. And since it is vacation time, I had some time to spend on it, leading to a release with some improvements that I’d like to present briefly.

One thing I noticed missing right away when I started to transcribe one of my mothers recipes was a segmented ingredients list. What I mean by that is the typical cake recipe that will say “For the dough…” “For the frosting…”

So I had to add support for this before I could continue with the recipe. The result looks like this:

Another weak point that became apparent was editing the ingredients on the edit page.  Initially, the ingredients list was just a plain text field. The previous release changed this to a list view, but the editing support consisted just of a popover with plain entries to add a new row.

This turned out to be hard to get right, and I had to go back to the designers (thanks, Jakub and Elvin) to get some ideas.  I am reasonably happy with the end result. The popover now provides suggestions for both ingredients and units, while still allowing you to enter free-form text. And the same popover is now also available to edit existing ingredients:

Just in time for the Christmas release, I was reminded that we have a nice and simple solution for spell-checking in GTK+ applications now, with Sébastian Wilmet’s gspell library. So I quickly added spell-checking to the text fields in Recipes:

Lastly, not really a new feature or due to my efforts, but Recipes looks really good in dark as well.

Looking back at the goals that are listed on the design page for this application,  we are almost there:

  • Find delicious food recipes to cook from all over the world
  • Assist people with dietary restrictions
  • Allow defining ingredient constraints
  • Print recipes so I can pin them on my fridge
  • Share recipes with my friends using e-mail

The one thing that is not covered yet  is sharing recipes by email. For that, we need work on the Flatpak side, to create a sharing portal that lets applications send email.

And for the first goal we really need your support – if you have been thinking about writing up one of your favorite recipes, the holiday season is the perfect opportunity to cook it again, take some pictures of the result and contribute your recipe!

 

css.php