This is part 1 of a mini-series. Part 2, Part 3.
Re-publishing, the original post is too old to show up on Planet GNOME at this point. It was published on August, 8th initially.
I’m a big fan of responsive touchpad gestures. For the last half a year (mostly January, February and during the summer) I’ve been working on improving gestures in many areas throughout GNOME. In this series I will do a (belated) overview.
Back/Forward Swipe
Late in the 3.32.x cycle, I saw a commit by Jan-Michael Brummer adding a back/forward swipe to Epiphany. It was really nice to finally have gestures, but it didn’t have any visual feedback. Less importantly, the direction was reversed, as if when scrolling with Natural Scrolling being off. I wanted to give a shot at improving it.
A proper gesture would have to “stick to finger”, showing screenshot of the previous or next page during the gesture, more or less what Safari does on macOS. Specifically, Epiphany would have to take screenshot of every page that is added into back/forward history, show it while the gesture is performed, then continue showing it until the next page loads enough to replace it. Unfortunately, this isn’t really possible to achieve in Epiphany itself: while WebKit does provide API to take snapshots, there’s no way to know when the previous/next page has loaded “enough”.
So I started looking into WebKit instead, where I found out that Safari’s gesture is actually implemented right there! Most parts are present, but the code was not cross-platform. So I started slowly adapting it for GTK. For the most part, the reusable code was a large part of the back end: page snapshot store and snapshot removal logic. That code is now shared between the platforms. The other parts, like actual event processing and drawing, had to be written from scratch.
One interesting detail about the gesture is that it doesn’t actually use gesture events! Libinput defines swipe gestures as synchronous movement of three or more fingers in the same direction. Mac gesture API is even more strict: it’s three fingers only, with four-finger swipes being reserved for the OS. But the gesture uses two fingers, how is this possible? Turns out it actually uses scroll events instead. (That’s also why it works with Magic Mouse in macOS, even though the code does not special-case anything for it)
When using scroll events, one has to be very careful. Touchpads generate scroll events continuously, in GTK it means that these gestures have GDK_SCROLL_SMOOTH scroll direction. At the very end of the scrolling, there will be a special event with is_stop field set to TRUE, which is used as a signal to start kinetic scrolling or, in our case, to end a swipe.
But there are other input devices, for example, mice. Most mice have “clicky” wheels that generate scroll events with direction instead of deltas. These events are impossible to use for swipes, so there’s no point in even trying to handle them. But there are also mice with freely scrolling wheels which generate the same events as touchpad, except there’s no event with is_stop == TRUE at the end. This means that they can be used to start a swipe, but it will get stuck as soon as the wheel stops. So, these mice have to be skipped too. Then there are touch mice where scrolling probably works same as on touchpad. I suspect swiping can work very well with them, same as it does with Magic Mouse on macOS, but there’s no way to distinguish these kinds of mice, at least as far as I know, so I had to disable it for any mice.
Another problem is that in order to not interfere with actual scrolling, the gesture controller must check whether it is possible to scroll the page first. Maybe there’s still space to scroll, maybe the page intercepts the scroll events. Then there has to be a threshold so that it’s hard to accidentally trigger the gesture. Thankfully, this part is shared with the Mac gesture. :)
A side effect of using scroll events is that this gesture still works with X11 and/or older semi-mt touchpads for which Libinput normally does not support any gestures.
So, now the gesture sticks to fingers. But there’s still an important bit missing: a snap-back animation. In order for a gesture to feel natural, it should snap back smoothly as soon as you lift your fingers, respecting the momentum. This was a lot easier to do than I expected, and after a few iterations applying Tobias Bernard’s suggestions I had an animation that I’m very satisfied with. How it works:
- The animation uses easeOutCubic interpolation
- Duration is naturally calculated as remaining distance divided by velocity, or if the velocity is 0, by a constant value instead
- After that, duration is multiplied by 3, matching easeOutCubic derivative at t=0. This ensures that initial velocity is same as it was before lifting the fingers
- Finally, the duration is clamped into [100ms, 400ms] range. This ensures that it’s never too slow or too fast, while still allowing it to respect momentum when possible
- If the page was swiped less than halfway through the window, there’s a small velocity threshold. If fingers are lifted when not moving, the gesture will be canceled and the page will smoothly slide back. On the other hand, if the page was swiped more than half way through, just lifting the fingers would finish the gesture, and one has to specifically flick back to cancel it
- If one starts swiping again while the animation is going, it will actually be stopped. This allows to continuously grab and release the page
Interestingly, Mac has a helper function that takes care of all this, and WebKit makes use of it.
Finally, the gesture should look nice. On Mac it uses CoreAnimation for drawing; WebKitGTK has to use Cairo. Since I didn’t have any mockups to work with, I reused Apple’s visuals, consisting of a dimming layer and a long subtle gradient for drop shadow, intending to replace them with something else later. But everybody whom I showed it liked it, so I left it as is.
The end result is that since version 2.24.0, WebKitGTK optionally supports 2-finger swipe gestures on touchpad.
Unfortunately, I was a little too late to enable it in Epiphany 3.32.0, but it was merged into 3.32.1 nevertheless. In 3.34.x, Yelp and Devhelp will also support it. Additionally, it’s enabled in Eolie and Odysseus browsers.
A bit later I also added touchscreen support. That was easy, because WebKit literally generates scroll events for touch scrolling, so it was simply a matter of feeding those events into the gesture controller. Additionally, I had to implement canceling, as all touchscreen gestures have to support it. Since scrolling on touchscreen is 1-finger swipe, the gesture is performed the same way.
This will be available in upcoming WebKitGTK 2.26.x, corresponding to GNOME 3.34.
This can be very disruptive feature in many cases, such as authentication widgets, so applications wanting to use it have to opt in by changing the value of this property.
Pinch Zoom
A smaller change was getting pinch zoom gesture to work on touchpads. Since this gesture was already available on touchscreens, it involved simply feeding touchpad gesture events into the gesture tracker, but the performance is severely lacking on heavy pages. Speeding it up is unfortunately still above my skill level. :)
Pinch zoom is enabled unconditionally, so it already works everywhere where WebKitGTK is used, including but not limited to Geary and documentation view in GNOME Builder.
I want to say thanks to the following people:
- Michael Catanzaro and Carlos Garcia Campos for code review and helping me with understanding WebKit codebase
- Tobias Bernard for testing and numerous design suggestions
- Jonas Dreßler for testing and feedback, especially on a touchscreen
Part 2 here.
