GNOME Shell saw many improvements during November. The commit log was dominated by cleanups, but a few improvements and polishments also found their way into the code.
The libcroco dependency was dropped by importing the source files into St. This is an important step in getting rid of libcroco, which is a dated CSS parsing library.
App Grid Improvements
The icon grid saw an important fix to dragging application icons. The icons were not properly being destroyed, and thus were piling up after dragging and dropping them over time. This fix was further improved to work on more situations. This set of fixes was backported to the 3.34 release.
A nice visual improvement landed on the page indicator of the icon grid.
For Mutter, November highlights were the introduction of regional clipping in Cogl, and big code cleanups.
Regional Clipping
When applications and GNOME Shell draw themselves, they communicate which parts of their contents changed. That information allows Mutter to submit only the changed contents to the monitor, which is an important optimization.
Example of GNOME Clocks being partially redrawn. The changed parts are painted in red.
Until GNOME 3.34, Mutter would calculate the bounding rectangle between all the regions that changed:
Mutter would submit the bounding box of all updated regions (in blue). In many situations, such as the above, that would include more than necessary.
This month, Mutter received the ability to update multiple regions independently, without using the bounding rectangle. In the example, Mutter now updates only what has actually changed:
The regions that Mutter submits (in blue) now matches the regions that really changed in the first picture (in red)
This yielded a significant improvement too! Under some circumstances, this change alone can reduce the time to submit frames by up to 44%.
Shadow Buffer
In some situations, in the native backend we now use a shadow buffer to render the stage off-screen before copying the content over to the actual buffer handed over to the display panel. While this may sound counter productive, it significantly increases performance from unusable to fairly pleasant on those systems that need it.
Other Highlights
We now prevent full window redraws when using dma-buf or EGLImage buffers on Wayland (mutter!948). This fixes partial updates of windows on Wayland, which can reduce the amount of data transferred between GPUs, CPUs, and the monitor. Together with the regional clipping explained above, this should significantly help saving battery.
Many, many Clutter and Cogl cleanups (mutter!921, mutter!819, mutter!933, mutter!932) landed too. These merge requests remove deprecated functions and features that, as time passes, are an increasingly burden for maintenance, and in some cases also prevent improvements and optimizations . About 28000 lines of legacy code has been cleaned out from Mutters own Cogl and Clutter versions so far, since we entered the 3.36 development phase. Extension authors, please make sure your extensions don’t use any of the removed code.
One legacy feature that dates back to when Clutter was a separate library used to write client applications was removed (mutter!911) from Mutter’s internal copy of Clutter. Not clearing the stage doesn’t make sense on a compositor.
Xwayland games that run fullscreen and change resolution should behave better now (mutter!739).
The last GNOME release, named “Thessaloniki”, was busy for GNOME Shell and Mutter. Many changes, ranging from code cleanups to architectural changes to performance improvements to new features landed.
Let’s take a look at the major highlights for the GNOME 3.34 release.
GNOME Shell
JavaScript Updates
GNOME Shell uses GJS, the GNOME JavaScript engine, to run. With the latest updates to GJS, such as the JS60 migration, GNOME Shell saw important updates making the codebase use modern tools and code practices.
Implicit Animations
One of the most important improvements that were part of GNOME Shell 3.34 was the transition to Clutter implicit animations. So far, GNOME Shell has been using the Tweener framework, which is written completely in JavaScript and, as a consequence, required extra work to communicate the animations to Clutter.
Using the implicit animations framework provided by Clutter allows GNOME Shell to skip a lot of JavaScript → C jumps, and thus, take less resources to run animations.
CI Integration
Testing GNOME Shell with CI is slightly tricky due to it and Mutter always being in lockstep. It is common that GNOME Shell from the git master branch needs Mutter from git master as well. Finding a good way to handle that prevented CI from landing earlier, but during this cycle we crafted custom container images and wrote the necessary scripts to allow for that.
Now, when running CI, GNOME Shell is tested with Mutter in tandem. Furthermore, on Mutter side, we’ve added a CI step to build and test GNOME Shell as well, catching many errors that would otherwise take some time to be found.
New Extensions Tool
GNOME Shell 3.34 also ships a new gnome-extensions tool, making it easier to create and manage GNOME Shell extensions. Due to the automation this new tool provides, such as extension templates, bash completion, packing, and testing extensions, it should be a significant improvement for extension authors.
Accessibility
With the ongoing transition to Wayland, GNOME Shell is taking ownership of many features that required random applications have full access to windows and their contents. One area where this bad practice is recurrent is the accessibility stack.
With GNOME Shell 3.34, a big coordinated effort was put in filling the missing accessibility bits when running a Wayland session, such as Locate Pointer, Click Assist, among others.
Click Assist menu. Courtesy of Olivier Fourdan.
Click Assist pie animation. Courtesy of Olivier Fourdan.
Folder Management
Last but not least, in GNOME Shell 3.34, it is possible to create, rename, and delete folders using Drag n’ Drop actions.
GNOME Shell 3.34 in Numbers
GNOME Shell 3.34 had 309 files changed, with 50,022 lines added and 26,924 removed. 59 merge requests were merged, and 1 was closed. 71 issues were closed by commits.
Sysprof is GNOME’s profiling framework, built on top of the `perf` API. With the GNOME 3.34 release, Mutter now integrates with Sysprof and is able to profile a lot of its internals.
Profiling Mutter is an important requirement to detect potential bottlenecks in the algorithms and the codebase, and have a proper understanding of how, where, and why things are the way they are.
GPU Hotplug
Mutter is now able to detect and use GPUs, usually external, that are plugged while the GNOME session is running.
KMS Transactions
In order for Mutter to support more complex hardware capabilities it needs to know how to utilize the atomic KMS kernel API. A first step in this direction was taken this cycle by implementing an internal transactional KMS API. Right now this API is backed by a non-atomic implementation, in order to make sure that we don’t break anywhere we currently fully function on, but in the future having this API will enable us to have mutter utilize more advanced hardware capabilities such as overlay planes and framebuffer modifiers. It is also a big step closer to being able to isolate KMS interaction into a dedicated thread eventually enabling low latency input device feedback once we move input processing into its own dedicated thread as well.
Improved Frame Scheduler
The algorithm that Mutter uses to schedule when to update the contents of the
monitor was fine-tuned and is now able to deliver new frames more smoothly. The new algorithm gives applications more time to draw their contents and notify Mutter about it. As a consequence, Mutter is much more likely to update frames in time for the monitor to display.
NVIDIA on Xorg frame throttling changes
Further in the past, Mutter used glxSwapBuffers() to throttle frame drawing when using the proprietary NVIDIA driver. This caused issues, as it’d block the main thread for long period of times if there was constantly new frames being scheduled. To mitigate this, Mutter developers introduced functionality imitating the GLX_INTEL_swap_event extension used by the Intel driver to make it possible for swapping buffers asynchronously, by receiving a “completion event” (swap event) later on.
This was done using NVIDIA specific GLX extensions combined with running a separate thread, where the “swap event” was generated, in contrast to when using GLX_INTEL_swap_event, where the Intel driver itself did this.
In practice, this caused issues, as it the relevant GLX extension implementation in the NVIDIA driver tended to be CPU heavy. However, due to the changes this cycle to our frame scheduling algorithm, the initial problem is now mitigated by scheduling frames in a way that we will we avoid blocking on glxSwapBuffers() for as long, meaning we could remove the CPU heavy “swap event” imitation code. This should mean less CPU usage when using the NVIDIA driver and displaying an application that continuously renders frames.
Graphene
Graphene is a library implementing complicated math functionality related to vertices, matrices, rectangles and other types usually related to 3D programming. In order to decrease maintenance burden on Mutter, which had its own set of complicated math functionality doing practically the same thing, we went and replaced most of that code, and switched to using Graphene directly. Not only did we decrease the amount of code we have to maintain, we also got all the fancy new improvements done to Graphene that we would otherwise be without!
For GNOME 3.34, all but matrices were converted to the corresponding Graphene types. That’s because Cogl and Graphene matrices have different semantics and, thus, will require more scrutiny.
XWayland on Demand
Mutter and GNOME Shell 3.34 earned the ability to run as a pure Wayland compositor, while still providing seamless compatibility with X11 by starting XWayland when required by applications. This is the culmination of much prior effort at refactoring internals and ironing out indirect X11 dependencies throughout Mutter and GNOME Shell specifically, and the GNOME session generally.
Being able to start Xwayland and other related X11 services on demand for legacy applications came out organically, and demonstrates we are no longer tied by our legacy support.
This feature is disabled by default, and can be enabled by adding ‘autostart-xwayland’ to `org.gnome.mutter experimental-features`.
Real-Time Scheduler
When running as a Wayland compositor and display server, it is important that Mutter keeps its reponsiveness all the time. However, Mutter runs as a regular user application, and is subject to starvation in various scenarios, such as intensive I/O.
Mutter 3.34 now requests real-time priority, which gives it more priority than
regular applications. This avoids stalls and stutterings when running GNOME Shell as a Wayland compositor.
This feature is disabled by default, and can be enabled by adding ‘rt-scheduler’
to `org.gnome.mutter experimental-features`.
MetaShapedTexture as a ClutterContent
MetaShapedTexture represents the visual contents of an application’s surface. It takes care of making sure the content is cropped and scaled properly, and tries to optimize drawing depending on what is opaque, and what is not.
ClutterContent, on the other hand, is Clutter’s way of defining deferred rendering instructions, also known as render nodes. This enables more efficient batching of draw calls, compared to the implicit batching implemented by the Cogl journal. This cycle we took the MetaShapedTexture, which was a ClutterActor subclass manually issuing Cogl drawing instructions, and turned into a set of fully deferred rendering instructions as defined by ClutterContent.
This not only improved batching of draw calls but greatly decreased the size of the Clutter actor tree, giving us a significant performance boost.
Here’s a quick comparison:
Turning MetaShapedTexture into a ClutterContent implementation (green bars) reduces frame times when many windows are open, compared to the previous implementation (yellow bars). Courtesy of Jonas Dreßler.
Mutter 3.34 in Numbers
Mutter 3.34 had 766 files changed, with 34,249 lines added and 37,268 removed. 74 issues were closed by commits.
| 122 | Jonas Ådahl |
| 109 | Carlos Garnacho |
| 82 | Marco Trevisan (Treviño) |
| 52 | Olivier Fourdan |
| 48 | Georges Basile Stavracas Neto |
| 40 | Florian Müllner |
| 30 | Adam Jackson |
| 27 | Daniel van Vugt |
| 24 | Hans de Goede |
| 24 | Robert Mader |
| 19 | Jonas Dreßler |
| 16 | Niels De Graef |
| 16 | Pekka Paalanen |
… a wise someone once muttered while walking on a beach, as they picked up a shell lying on the sand. Indeed, every shell began somewhere, crossed a unique path with different goals and driven by different motivations. Some shells were created to optimize for mobility; some, for lightness; some, for speed; some were created to just fit whoever is using it and do their jobs efficiently. It’s statistically close to impossible to not find a suitable shell, one could argue.
So, is this a blog about muttered shell wisdom?
In some way, it actually is. It is, indeed, about Shell, and about Mutter. And even though “wisdom” is perhaps a bit of an overstatement, it is expected that whoever reads this blog doesn’t leave it less wise, so the word applies to a certain degree. Evidently, the Shell in question is composed of bits and bytes; its protection is more about the complexities of a kernel and command lines than sea predators, and the Mutter is actually more about compositing the desktop than barely audible uttering.
What Now?
For a long time, the development of Mutter and GNOME Shell was surrounded by silence. This blog is a humble attempt to bring those two critical components of the GNOME desktop to the spotlight, even if only a tiny bit more.
It would be naive to say that posts will be published in a consistent frequency, but the initial goal is monthly development reports and eventual one-shot deep dives into various bits and pieces of these two components.
