I wanted to start this blog post with “It’s that time of year again”, but looks like Michael beat me to it. So, let’s take a look at some of the changes in GNOME Games 3.38:
retro-gtk 1.0
The library Games uses to implement Libretro frontend, retro-gtk, has been overhauled this cycle. I’ve already covered the major changes in previous blog post, but to recap:
Cores now run in a separate process. This provides a better isolation: a crashing core will display an error screen instead of crashing the whole app. This can also improve performance in case the window takes a long time to redraw, for example with fractional scaling.
Libretro cores that require OpenGL should now work correctly.
The core timing should now be more accurate.
Fast-forwarding should actually work, although we don’t make use of it in Games at the moment.
Finally, retro-gtk now has proper docs, published here to go along with the stable API.
Nintendo 64 support
This was on the radar for a while, but wasn’t possible because all available Nintendo 64 cores use hardware acceleration. Now that retro-gtk supports OpenGL, we can ship ParaLLEl N64 core, and it just works. There’s even a menu to switch between Controller Pak and Rumble Pak.
Unfortunately, retro-gtk still doesn’t support Vulkan rendering, so we can’t enable the fast and accurate ParaLLEl RDP renderer yet.
Collections
Neville Antony has been working on implementing collections as part of his GSoC project. So far we have favorites, recently played games and user-created collections.
In my previous blog post I already mentioned some improvements to the app startup time, but a noticeable progress was made since then. Newly added aggressive caching allows the app to show the collection almost instantly after the first run, and then it can scan and update the collection in background.
Search provider
Having a complete cache of the collection also allowed to easily implement a fast search provider, so the collection can now be searched directly from GNOME Shell search.
Nintendo DS screen gap
Nintendo DS has two screens. Most games use one of the screens to display the game itself, and the other one for status or map. However, some games use both screens and rely on the fact they are arranged vertically with a gap between them.
If the two screens are shown immediately adjacent to each other, these games look confusing, so in 3.38 a screen gap will be automatically added when using vertical mode.
Before
After
The size of the screen gap is optimized for a few known games, such as Contra 4, Sonic Colors or Yoshi’s Island DS, other games use a generic value of 80 pixels.
If you know another game that needs to use a specific gap size, please mention it in the comments or open an issue on GitLab.
Gamepad hot plugging in Flatpak
Previously, a major limitation of Flatpak version was that gamepads had to be plugged in before starting the app. Thanks to a change in libmanette 0.2.4, this has been fixed and gamepads can be connected or disconnected at any time now.
Technically, this isn’t a 3.38 addition, and in fact the 3.36.1 build on Flathub already supports it. Nevertheless, it was done since my last blog post and so here we are. :)
Miscellaneous changes
Game covers now look prettier thanks to Neville: they are now rounded and have a blurred version of the cover as background.
The preferences window has been overhauled to be more similar to the libhandy preferences; unfortunately we can’t use HdyPreferencesWindow yet because of some missing functionality.
When testing or re-mapping a controller, analog stick indicators now show the stick’s precise position instead of just highlighting one of the edges.
Veerasamy Sevagen implemented an empty state screen for the collection search.
Swipe gestures to go back are now supported throughout the app. The only place that lacks the gesture now is exiting from a running game, because the game may require pointer input, and having a swipe there may cause conflicts.
Getting Games
As always, the latest version of the app is available on Flathub.
Yesterday, I released GNOME Games 3.35.90, so we’re in feature freeze for 3.36.0. Let’s take a look at the changes during the 3.35.x cycle:
Faster collection loading
For a long time, Games loaded collection asynchronously using Vala async functions. While it didn’t block the UI completely, it was still slow and caused frequent UI stalls until it loaded completely. In 3.36, collection loading uses a separate thread instead and is noticeably faster as a result, while the UI is perfectly smooth the whole time.
Cover loading has been moved to a thread as well, so both initial loading and scrolling while covers are loading should now be fast and smooth.
There’s still lots of room for improvement, but for the moment this improves things somewhat. ?️
Steam integration improvements
New-style vertical covers are now supported for Steam games.
Additionally, Steam tools such as Proton and Steamworks Common Redistributables don’t show up as games anymore.
Restarting games
Veerasamy Sevagen added a way to restart games without exiting them.
Other changes
Savestates were renamed to snapshots, and backup/restore was renamed to export/import.
Nightly and development builds have a new icon made with App Icon Preview.
Lists now look more consistent with each other on mobile:
On desktop, the first two lists turn into sidebars and look mostly the same way as before, while other lists will still have rounded corners and separators.
And finally, there has been a lot of refactoring and code cleanups.
And that’s it? If the changelog for 3.36 seems a bit short, it’s because it is. Due to unfortunate timing, a lot of work planned for 3.36 has been postponed to 3.38 so that we get more than a few weeks before feature freeze to finish and test it. ?️ So, let’s take a look at what retro-gtk 1.0 will bring.
retro-gtk 1.0
API break
First of all, retro-gtk 1.0 will not be API-compatible with the previous versions. In some cases the API is just simplified or made nicer to use (for example, RetroMainLoop is gone, having been merged into RetroCore), but a lot of changes have a very specific reason, more on that later. Since there will be more changes down the road, there’s no point in describing the API changes just yet, so instead let’s take a look at the bigger features:
More precise timing
Currently, retro-gtk uses g_timeout_add() for doing the main loop in the game. It’s… Not very precise. Instead, we have a custom GSource now implementing a more precise timer that allows an error of just a few microseconds instead of a few hundred.
Of course, it’s still not synced to the GTK refresh rate and so occasionally results in dropped frames. I’m not particularly happy with that, but that’s for later.
Audio playback improvements
With such an imprecise timing, it would be expected that the games would run at a wrong speed. And sometimes they do, but more often than not they still work fine. This happens because audio playback is blocking (each call waits until the previous audio has finished playing) and so the game can never run faster than its audio can play.
And as long as games output audio once per frame, it’s not an issue. However, libretro defines 2 callbacks for audio: one sends 2 samples, one for each channel, and the other one sends a batch of arbitrary length. And of course a lot of cores use the former.
At first retro-gtk just played samples immediately as they arrived. Of course, the cores using the former way were slowed down to a crawl because each call waited for the previous samples to play.
A year ago Adrien added a workaround after a report that a core was slowing down by batching up to 512 samples and playing them all at once. This solved the immediate problem, because it meant cores could do many consecutive calls sending samples and not get slowed down. However, outputting more than 512 samples at once still caused slowdowns (a lot of cores output 1000 or more samples per frame), and it also means that if a core sends a number of samples that isn’t power of 2 per frame, or if it sends a different amount of samples on every frame, there will be inconsistent slowdowns or jittery audio.
Initially I opted for making audio playback threaded and queueing any additional samples to resolve this. It worked, but led to some subtle desync issues if the game ran a little faster. It could be as subtle as 6 extra audio frames (12 samples) every 22 seconds (0.45% assuming the framerate is 60 fps), but it was still noticeable, so instead we now queue any samples sent during the frame and then play them once at the end of every frame. While this is not perfect (if the core sends a different number of samples, it’s still possible to get slowdowns after a “long” frame), it works and it solves slowdown in cores like PX68K.
Fast-forwarding that actually works
While it might be good that audio playback naturally prevents games from running too fast, it also means that the speed rate property (unused in Games currently) did nothing when trying to speed the game up rather than slow it down. To solve this, retro-gtk now resamples audio to match speed rate using libsamplerate. Now setting speed rate to values larger than 1 works, although with a “chipmunk effect”, as resampling also changes audio pitch. ?️
Running cores in a separate process
This is a big change, and it’s something I’ve started working on not long after 3.34.0 release, though at one point put it into a hiatus and then resumed. It means moving the game logic into a separate process and only having a proxy that sends input and receives output in the UI process. It’s similar to what web browsers have been doing long before, with separate web processes for each tab.
Why?
Many reasons. For one, crash resilience. If a core crashes, currently the whole app is taken down. When the core runs in its own process, instead Games can show an error and offer to restart the game.
It ensures there are no UI stalls due to a slow core. Running multiple cores at the same time will start a separate runner process for each core, which means the runner process side code can assume there’s only ever one core and can be simplified a lot. It improves performance when drawing the game into the widget is slow, like with fractional scaling, as drawing now happens asynchronously and so even if the UI process can’t keep up drawing everything in time, the game still runs at full speed. After a core is stopped, it’s cleanly uninitialized because its process exits. And it also allows us to implement some new features, but more on that later.
How?
The implementation is very much inspired by Christian Hergert’s GUADEC 2019 talk and by Builder’s git plugin and Sysprof. The runner process helper is a separate lightweight binary called retro-runner that isn’t linked to retro-gtk or even to GTK.
For communication it uses both messaging and shared memory. Calls such as switching disks, saving/loading states, pausing and resuming use messaging, specifically D-Bus over a private socket connection. This makes it possible to use gdbus-codegen for generating boilerplate, though there’s still some boilerplate wrapping those calls to expose in a public API. It also allows to pass file descriptors, which are useful for…
…shared memory, specifically a memfd that is passed to the other process and mmap-ed on both sides, which is used for input and video.
This required some changes in how input works: libretro input is polling-based and retro-gtk follows that. However, in every single RetroController implementation retro_controller_poll() is no-op and instead the controller gets the input state on its own via signals, stores it and returns it on demand. With that in mind, poll() was removed and instead controller implementations can now notify about their state changes. When that happens, their complete state is serialized and written into a shared memory block. Since the complete state for one controller, including mouse, pointer, lightgun, gamepad and keyboard state is under 1kb, it’s fast enough to just write the whole thing and not only differing parts. Then, when the core polls input state, a copy of the contents of shared memory is made and then any queries return values from that copy. This means the input is still fast and has no noticeable latency increase over running in the same process, and at the same time we follow the spec closer because input is now actually polling-based on the runner process side, and asking for input state before polling it does actually return the previous state.
Additionally, the call to set controller rumble state previously returned a boolean value to indicate whether it was succcessful or not. That was previously exposed in RetroController as is, but calling it in sync just to get that value isn’t really an option. So instead there’s now a separate retro_controller_get_supports_rumble() call and retro_controller_set_rumble_state() returns nothing, allowing it to be called via D-Bus without degrading performance.
Video is passed similarly to input: the shared memory contains a framebuffer that runner process writes to and UI process reads from. Unfortunately, it’s not very efficient, because libretro gives us a preallocated framebuffer, resulting in a copy on the runner process side. It’s even worse on the UI process side, as there’s a copy (which is unnecessary, but I haven’t got to removing it yet) + uploading the texture to GPU to render it. And while libretro provides a callback to get an address of our framebuffer, retro-gtk doesn’t implement it yet and almost no core uses it. ?️ Thankfully, the resolution is usually pretty low (160×144, 240×160, 320×240, or sometimes 640×480), so this pipeline works, and it’s even noticeably faster than it was in a single process for me.
The problematic part is that actually telling UI process to redraw is currently done via a message. ?️ While it works, it’s extra latency that could be avoided. But there’s still time to change it.
OpenGL core support
One feature that we’ve wanted for a long time, but could never implement is supporting libretro cores that use hardware rendering. It should be simple, right? After all, we already use OpenGL in the widget that draws the game.
In the most basic form, the core sends us specs of a context and a framebuffer (API, version, framebuffer parameters such as whether it has depth/stencil buffer). We get a context from somewhere as asked and provide the framebuffer as needed. Additionally we’re expected to provide pointers to the GL functions by name.
The only quirk is that most cores want a compatibility profile context. And I couldn’t achieve it no matter what I did. Either it corrupted GTK state or eglCreateContext() created a core profile context even though I asked for compatibility profile.
Subprocess comes to the rescue! With that I can easily have whatever context the core wants on the runner process side and it just works. Hence:
Nintendo 64, via ParaLLEl N64 core
Dreamcast, via Flycast core
With OpenGL cores working, we can run games for a lot of platforms we don’t currently support, such as Nintendo 64 and Sega Dreamcast. While I can’t say yet which platforms will be supported in 3.38, I’m pretty sure Nintendo 64 will make it. Less sure about Dreamcast, because while Flycast core runs pretty well, it has some quirks. For example, if a state was saved without a controller plugged in, it won’t ever see the controller after restoring that state again, and for some reason Sonic Adventure defaulted to Japanese language. More importantly, Dreamcast game detection in Games isn’t very good right now and needs improvements before it’s useful.
Additionally, for Nintendo 64 I implemented a simple controller expansion switcher, because with Nintendo 64 you have to choose between reliably saving games and having rumble. ?️
At least we can automatically disable rumble for controllers that don’t support it, such as the keyboard assigned for player 2 in the screenshot. The UI needs more work too, for example, I’m not particularly happy about Player 1/Player 2 labels, but I needed something for testing. ?️
So, is it done?
Not yet. While OpenGL support works pretty well, there are other things to do, such as Vulkan support, as both of these cores can use it. That’s going to be interesting, because I have absolutely no experience working with Vulkan, unlike with OpenGL. Looking forward to learning it, and looking forward to an awesome 3.38 release, even though we still haven’t released 3.36 yet. ?️
A year ago, Adrien Plazas stepped down as a maintainer, so Games 3.32.0 was released without an accompanying blog post, since I didn’t have a blog at the time. Now it’s time to make up for it with a blog post about 3.34.0.
GSoC and savestates
As part of his GSoC project, Andrei Lişiţă implemented a savestate manager.
Savestates are a common feature in game emulators, that work similarly to snapshots in virtualization: emulator takes a full snapshot of RAM and storage, which can be loaded later to restore the game to the same exact state it was in when saved.
The app has supported savestates for a long time: when you exit a game, a savestate is created. Then when you run it again, Games offers to restore that savestate or reset the game. However, there was no way to manage savestates during the game, or to have more than one savestate at a time.
Now we have a shiny new sidebar for managing savestates and can save and load them on demand. This can be used for saving memorable moments in games, or for cheating your way through difficult games via saving and reloading every time you make a mistake.
There is still room for improvement, for example, there is no way to use savestates with gamepad right now.
Since directory layout is different now, existing data is automatically migrated on the first run. This is a one-way process, downgrading to 3.32.1 after running 3.34.0 is not possible!
Nintendo DS screen layouts
GNOME Games supports running Nintendo DS games since 3.30. However, the system’s two screens make it awkward to play without having a screen in portrait orientation. While ideally we want retro-gtk to support support rearranging screens, for now I implemented this using options of DeSmuME core:
So far we support a vertical mode, two horizontal modes and a single screen mode, corresponding to DeSmuME’s top/bottom, left/right, right/left and quick switch modes. While the first 3 modes work exactly the same as in RetroArch, single screen mode has some improvements: we remember and restore the currently viewed screen, provide a visible button for switching between the screens and have separate keyboard shortcuts for that (it’s using top only and bottom only internally, not quick switch).
Naturally, a downside is that it only works with DeSmuME and DeSmuME 2015 cores.
At first it was a global setting. This led to some problems, such as squished screenshots when loading the game if the current layout doesn’t match the one the game was played with the last time, which was made even more apparent after Andrei’s savestate work landed, showing the squished screenshots even more prominently.
Because of that, I reworked the feature to store the screen layout inside savestate metadata instead of a global setting. While this required a fix in retro-gtk, screen layout now can be different not just for every game, but even for every savestate, meaning previews always perfectly match the loaded game.
Platforms instead of Extensions
For a long time, Games had an “Extensions” page in preferences, showing all the installed plugins. This was one of the more confusing parts of the app:
There are many problems with this page: many people assumed that the listed platforms were the only platforms we support, even though there are 24 more platforms that are supported directly, not via a plugin. Additionally, “platform support”, via plugin or not, means that Games can list games from this platforms in collection view. It does not automatically mean that the app can run those games, that would also require the corresponding libretro core to be installed. And even then, some games need firmware to run. None of this is really obvious from the page.
All in all, this means that Games can run NES and Game Boy games, even though they aren’t listed on the page, and at the same time it cannot run DOS or Sega Saturn games, even though they are listed, which makes no sense unless you’re involved in development of the app.
Even more confusingly, the modules that the page lists are actually called “plugins” internally, not “extensions”.
To solve these problems, this page has been removed and replaced with a Platforms page that lists supported platforms, lists the currently used core for libretro platforms and grays out the platforms without cores, so that it’s visible which games can and cannot be run.
While our official build on Flathub ships only one core per platform, it’s possible to have multiple cores for a single platform. In that case, the page will allow to choose the core to run the games with. I hope the new page will be a lot more useful.
Backup and Restore
Another feature that landed during this cycle is backing up and restoring for savestates, courtesy of Adwait Rawat.
These changes include a page in preferences that allows to export all the savestates as one archive, or to restore an existing backup. This can be useful, for example, to move data to a different device.
Goodbye, developers view
Developers view was implemented along with Platforms view by Saurabh Sabharwal as part of his GSoC 2018 project. While Platforms view was very successful, Developers view used thegamesdb metadata, which wasn’t very reliable. Often, it mixed developers and publishers, or duplicated the same developer multiple times with slight variations. And then, thegamesdb changed its API, so for the last few months, this view has been completely empty. Because of that, I went ahead and removed this view:
Adaptive UI
I’ve been slowly making the UI adaptive for the last year. 3.34.0 finally marks the point where the app can run on a phone:
Main screen
Platforms
Platform details
Search
In game
Savestates
Preferences
Backup & Restore
However, there is still a very important bit missing: touch controls to actually play games without a gamepad or keyboard.
Better Fullscreen
In 3.32.1, headerbar in fullscreen is hidden, but shows up on any cursor movement. This can be annoying when playing a mouse-heavy game (for example, on Nintendo DS), so in 3.34.0 the headerbar can only be revealed by pushing the top of the screen, similar to the behavior in Epiphany and other apps.
Miscellaneous Changes
Andrei added an error message that shows up after opening a non-game file:
Media switcher has a dropdown arrow now:
Cursor now autohides in windowed mode after 3 seconds of inactivity, just as in fullscreen.
Game covers aren’t blurry on HiDPI screens and aren’t darkened anymore.
Thanks to all the contributors who made this release possible!
Getting Games
As always, the latest version of the app is available on Flathub.
GNOME Games has been participating in Google Summer of Code for many years, and this one is no exception. This time Andrei Lişiţă a.k.a. Yetizone was implementing a savestate manager.
Andrei’s work involved redoing $XDG_DATA_HOME/gnome-games/ directory layout, writing a migrator for existing data, reworking the app to support having multiple savestates at once, implementing on-demand loading and saving, and implementing the UI.
Check out his blog posts for more information about the project:
See also his lightning talk at GUADEC 2019 about the project! I couldn’t attend it, but I watched the livestream. :)
It was my first year as a mentor. It did feel a little weird, since I never participated in GSoC as a student, and I’m also a student myself. Nevertheless, the project was finished successfully, so we have a shiny new feature. Most of the work is already merged and will be available in GNOME Games 3.34.0. The two remainingchanges will be landed early in 3.36.x cycle.
Andrei is awesome to work with. He started coding early in community bonding period, and needed next to no handholding, so for the the most part of the coding period my involvement was simply answering questions and reviewing code, later testing and reporting bugs.
While the project was finished, I think there are things I think I could do a lot better as a mentor:
While we were focused on the technical side of the project, I didn’t pay enough attention to the social side. Namely, I didn’t track GUADEC announcements closely (since I couldn’t attend it this year anyway), so Andrei almost ended up missing it. Thanks to Gaurav Agrawal who helped Andrei with registration and requesting travel sponsorship.
Until the last few weeks, the project consisted of a huge merge request which quickly got unwieldy. For planning we used Matrix. I saw other people using more sophisticated workflows for GSoC, such as networks of small merge requests and issues in interns’ forks on GitLab, which can be reviewed and merged one by one, I think we should have used a workflow like that as well.
I ended up postponing 3.33.90 release for a week, merging many followup changes during the freeze, and even then merging changes up until 3.33.91 (Since Games is not a core app, we are allowed to do that, but usually we try to follow the freezes nevertheless). This probably wasn’t a good thing to do and I apologize to the translation teams. :(
All in all, I want to thank Andrei for the amazing contribution and for being an awesome person in general. :)
