Hackfest – Space and Meaning

Local-First Workshop (feat. p2panda)

This week we had a local-first workshop at offline in Berlin, co-organized with the p2panda project. As I’ve written about before, some of us have been exploring local-first approaches as a way to sync data between devices, while also working great offline.

We had a hackfest on the topic in September, where we mapped out the problem space and discussed different potential architectures and approaches. We also realized that while there are mature solutions for the actual data syncing part with CRDT libraries like Automerge, the network and discovery part is more difficult than we thought.

Network Woes

The issues we need to address at the network level are the classic problems any distributed system has, including:

Discovering other peers
Connecting to the other peers behind a NAT
Encryption and authentication
Replication (which clients need what data?)

We had sketched out a theoretical architecture for first experiments at the last hackfest, using WebRTC data channel to send data, and hardcoding a public STUN server for rendezvous.

A few weeks after that I met Andreas from p2panda at an event in Berlin. He mentioned that in p2panda they have robust networking already, including mDNS discovery on the local network, remote peer discovery using rendezvous servers, p2p connections via UDP holepunching or relays, data replication, etc. Since we’re very interested in getting a low-fi prototype working sooner rather than later it seemed like a promising direction to explore.

p2panda

The p2panda project aims to provide a batteries-included SDK for easy local-first app development, including all the hard networking stuff mentioned above. It’s been around since about 2020, and is currently primarily developed by Andreas Dzialocha and Sam Andreae.

The architecture consist of nodes and clients. Nodes include networking, materialization, and an SQL database. Clients sign and create data, and interact with the node using a GraphQL API.

As of the latest release there’s TLS transport encryption between nodes, but end-to-end data-encryption using MLS is still being worked on, as well as a capabilities system and privacy-respecting deletion. Currently there’s a single key/value CRDT being used for all data, with no high-level way for apps to customize this.

The Workshop

The idea for the workshop was to bring together people from the GNOME and local-first communities, discuss the problem space, and do some initial prototyping.

For the latter Andreas prepared a little bookmark manager demo project (git repository) that people can open in Workbench and hack on easily. This demo runs a node in the background and accesses the database via GraphQL from a simple GTK frontend, written in Rust. The demo app automatically finds other peers on the local network and syncs the data between them.

Bookmark syncing demo using p2panda, running in Workbench

We had about 10 workshop participants with diverse backgrounds, including an SSB developer, a Mutter developer, and some people completely new to both local-first and GTK development. We didn’t get a ton of hacking done due to time constraints (we had enough program for an all-day workshop realistcally :D), but a few people did start projects they plan to pursue after the workshop, including C/GObject bindings for p2panda-rs and an app/demo to sync a list of map locations. We also had some really good discussions on local-first architecture, and the GNOME perspective on this.

Thoughts on Local-First Architectures

The way p2panda splits responsibilities between components is optimized for simple client development, and being able to use it in the browser using the GraphQL API. All of the heavy lifting is done in the node, including networking, data storage, and CRDTs. It currently only supports one CRDT, which is optimized for database-style apps with lots of discrete fields.

One of the main takeaways from our previous hackfest was that data storage and CRDTs should ideally be in the client. Different apps need different CRDTs, because these encode the semantics of the data. For example, a text editor would need a custom text CRDT rather than the current p2panda one.

Longer-term we’ll probably want an architecture where clients have more control over their data to allow for more complex apps and diverse use cases. p2panda can provide these building blocks (generic reducer logic, storage providers, networking stack, etc.) but these APIs still need to be exposed for more flexibility. How exactly this could be done and if/how parts of the stack could be shared needs more exploration :)

Theoretical future architectures aside, p2panda is a great option for local-first prototypes that work today. We’re very excited to start playing with some real apps using it, and actually shipping them in a way that people can try.

What’s Next?

There’s a clear path towards first prototype GNOME apps using p2panda for sync. However, there are two constraints to keep in mind when considering ideas for this:

Data is not encrypted end-to-end for now (so personal data is tricky)
The default p2panda CRDT is optimized for key / value maps (more complex ones would need to be added manually)

This means that unfortunately you can’t just plug this into a GtkSourceView and have a Hedgedoc replacement. That said, there’s still lots of cool stuff you can do within these constraints, especially if you get creative in the client around how you use/access data. If you’re familiar with Rust, the Workbench demo Andreas made is a great starting point for app experiments.

Some examples of use cases that could be well-suited, because the data is structured, but not very sensitive:

Expense splitting (e.g. Splittypie)
Meeting scheduling (e.g. Doodle)
Shopping list
Apartment cleaning schedule

Thanks to Andreas Dzialocha for co-organizing the event and providing the venue, Sebastian Wick for co-writing this blog post, Sonny Piers for his help with Workbench, and everyone who joined the event. See you next time!

GNOME 45 Release Party & Hackfest

In celebration of the 45 release we had a hackfest and release party in Berlin last week. It was initially supposed to be a small event, but it turns out the German community is growing more rapidly than we thought! In the end we were around 25 people, about half of them locals from Berlin :)

GNOME OS

Since many of the GNOME OS developers were in town for All Systems Go, this was one of the main topics. In addition to Valentin, Javier, and Jordan (remote), we also had Lennart from systemd and Adrian from carbonOS and discussed many of the key issues for image-based operating systems.

I was only present for part of these discussions so I’ll leave it to others to report the results in detail. It’s very exciting how things are maturing in this area though, as everyone is standardizing on systemd’s tools for image-based OSes.

Discussing the developer story on image-based OSes with Jonas and Sebastian from GNOME Shell/mutter. Left side: Adrian, Javier, Valentin, Sebastian. Right side: Jonas, Kai, Lennart

Local-First

On Saturday the primary topic was local-first. This is the idea that software should always work offline, and optionally use the network when available for device sync and collaboration. This allows for people to own their data, but still have access to modern features like multiplayer editing.

People in the GNOME community have long been interested in local-first and we’ve had various discussions and experiments in this direction over the past few years. However, so far we have not really investigated how we’d implement it at a larger scale, and what concrete steps in that direction would look like.

For context, any sync system (local-first or not) needs the following things:

Network: Device discovery, channel to send the actual data, way to handle offline nodes, encryption, device authentication, account management
Sync: Merging data from different peers, handling conflicts
UI: User interface for viewing and manipulating the data, showing sync status, managing devices, permissions, etc.

Local-first usually refers to systems that do the “sync” part on the client, though that doesn’t mean the other areas are easy :)

Muse

Adam Wiggins stopped by on Saturday morning to tell us about his work on Muse, a local-first whiteboard app for Apple platforms. While it’s a totally different tech stack and background, it was super interesting because Muse is one of very few consumer apps using local-first sync in production today.

Some of my takeaways from the session with Adam:

Local-first means all the logic lives in the client. In the Muse architecture, the server is extremely simple, basically just a dumb pipe routing data between clients. While data is not encrypted end-to-end in their case, it’s possible to use this same architecture with E2E.
CRDTs (conflict-free replicated data types) are a magical new advancement in computer science over the past few years, which makes the actual merging of content relatively easy.
Merge conflicts are not as big a deal as one might think, and not the hardest problem to solve in this space.
Local-first is a huge opportunity for desktop projects like GNOME. We were not really able to be competitive with proprietary software in the past decade on features like sync and multiplayer because we can’t realistically run huge cloud services for every single app/use case. Local-first could change this, since the logic is shifting back to the client. Servers become generic dumb pipes, which all kinds of apps can use without needing their own custom sync server.

To learn more about Muse, I recommend watching Adam’s Local-First Meetup talk from earlier this year, which touches on many of the topics we discussed in our hackfest session as well.

Other Relevant Art

The two projects we discussed as relevant art from our community are Christian Hergert’s Bonsai, and Carlos Garnacho’s work on RDF sync in tracker (codename “Emergence”).

Bonsai is not quite local-first architecturally, since it assumes an always-on home server. This server hosts your data and runs services, which apps on your other devices can use to access data, sync, etc. This is quite different from the dumb pipe server model discussed above, and of course comes with the usual caveats with any kind of public-facing service on local networks (NAT, weird network configurations, etc.).

Codename “Emergence” is a way to sync graph databases (such as tracker’s SPARQL database). It only touches on the “sync” layer, and is only intended for app data, e.g. bookmarks, contacts, and the like. There was a lot of discussion at the hackfest about whether the conflict resolution algorithm is/could be a CRDT, but regardless, using this system for syncing some types of content wouldn’t affect the overall architecture. We could use it for syncing e.g. bookmarks, and share the rest of the stack (e.g. network layer) with other apps not using tracker.

Afternoon local-first discussion. Left-to-right: Zeeshan, Sebastian, Andrei, Marvin, Adrian, Carlos, and myself

Next Steps

By the end of the hackfest, we had a rough consensus that long-term we probably want something like this:

Muse-style architecture with dumb pipe sync servers that only route encrypted traffic between clients
Some kind of system daemon that apps can use to send packets to sync servers, so they don’t all have to run in the background
The ability to fall back to other kinds of transport with full compatibility, e.g. local network or USB keys
A client library that makes it easy to integrate sync into apps, using well-established CRDTs

However, there was also a general feeling that we want to go slowly and explore the space before coming up with over-engineered solutions. To this end, we think the best next step is to try CRDTs in small, self-contained apps. We brainstormed a number of potentially interesting use cases, including:

Alarms: Make extra sure you hear your alarms by having it synced on all devices
Scratchpad: Super simple notepad that’s always in sync across devices
Emoji history: The same recently used emoji on all devices
Podcasts: Sync subscription list, episode playback state, per-episode progress, currently played episode, etc.
Birthday reminders: Simple list of birthdays with reminder notifications that syncs across all devices

For a first minimum viable prototype we discussed ways to cut as many corners as possible, and came up with the following plan:

Use an off-the shelf plain text CRDT to build a syncing scratchpad as a first experiment
To avoid having to deal with servers, do peer-to-peer transfer only and send data via WebRTC data channel
For peer discovery, just hardcode a public WebRTC STUN server in the client
Simple Rust GTK app mostly consisting of a text area, using gstreamer for WebRTC and automerge for CRDTs
Sync only between two devices

We’ll see how this develops, but it’s great to have mapped out the territory, and put together a concrete plan for next steps in this direction. I’m also conscious that we’re a huge community and only a handful of people were present at the hackfest. It’s very likely that these plans will evolve as more people get involved and we get more experience working with the technology.

For more detail on the discussions, check out the full notes from our local-first sessions.

If you’d like to experiment with this in your own app and have any questions, don’t hesitate to reach out :)

Transparent Panel

Jonas has had an open merge request for a transparent panel for a number of years, and while we’ve tried to get it over the line a few times we never quite managed. Recently Adrian Vovk was interested in giving it another try, so at the hackfest him and Jonas sat down and did some archaeology on Jonas’ old commits, rebased it, got it to work agian, and opened a new merge request.

Adrian and Jonas hard at work rebasing ancient commits

While there are still a few open questions and edge cases, it’s early in the cycle so there’s a real chance that we might finally get this in for 46 :)

And More!

A few other things I was involved with during the hackfest:

Julian looked into one of my pet bugs: The default generated avatars when you create a new account not looking like AdwAvatar, but using an older, uglier implementation. This is surprisingly tricky because GDM/GNOME Shell can’t show GTK widgets, and the exported PNG avatars from libadwaita can only be exported at the size they’re being displayed at (which is smaller than in GDM/GNOME Shell).
We worked a bit on Annotations in Evince with Pablo, and also interviewed Keywan about how they use annotations in the editorial process for their magazine.
DieBahn was officially renamed to Railway, and we discussed next steps for the app and train APIs in general. Railway works for so many providers by accident, because so many of them use the same backend (HAFAS), but it’d be great to have actual open APIs for querying trains from all providers. Perhaps we need a lobbying group to get some EU legislation for this? :)
We discussed a “demo mode”, i.e. an easy way to set up a device with a bunch of nice-looking apps, pre-loaded with nice-looking content. One potential approach we discussed was a script that installs a set of apps, and sets them up with data by pre-filling their .var/app/ directories. The exact process for creating and updating this data would need looking into, but I’m very interested in getting something set up for this, because not having it really our software hard to demo.
Marvin showed me how he uses CLion for C/Vala development, and we discussed what features Builder would need to gain for him to switch from his custom Vala setup in CLion to Builder.

Julian working on fixing the default avatars

Thanks to Sonny for co-organizing, Cultivation Space for hosting us, and the GNOME Foundation for financial sponsorship! See you next time :)

Berlin Mobile Hackfest

Last week we had a small hackfest in Berlin, with a focus on making GNOME work better on mobile across the stack. We had about 15 people from various projects and backgrounds attending, including app developers, downstreams, hardware enablement, UX design, and more. In addition to hacking and planning sessions we also had some social events, and it was an opportunity to meet nice people from different corners of the wider community :)

The Event(s)

The postmarketOS gang had their own hackfest over the weekend before ours, and on Monday we had the Cultivation Space venue for some chill random hacking. I wasn’t there for most of this, so I’ll let others report on it.

The main hackfest days were Tuesday and Wednesday, where we had an actual unconference agenda and schedule. We met at 11:30 in the morning at Cultivation Space and then managed to get through a surprisingly large list of topics very efficiently on both days. Kudos to Sonny for coming up with the format and moderating the schedule.

Late night podcast recording on Wednesday (note the microphone in the bottle in the center)

systemd and Image-Based OSes

On Tuesday Lennart from systemd stopped by for a bit, so we had a session with him and the postmarketOS people discussing integration issues, and a larger session about various approaches for how to do image-based OSes. It’s clear that this direction is the future — literally everyone is either doing it (Endless, Silverblue, GNOME OS, SUSE, Carbon OS, Vanilla OS) or wants to do it (PureOS, postmarketOS, elementaryOS). There are lots of different approaches being tried and nobody has the perfect formula for it yet, so it’s an exciting space to watch.

Edit: Clarification from postmarketOS: “We are looking into doing a version of postmarketOS that is image-based/has an immutable file system. But it will still be possible to use postmarketOS without it.”

I personally only half followed some of this while working on other stuff, but it was definitely interesting to hear people discuss the tradeoffs of the various approaches. For example, OSTree doesn’t support offline security which is why Lennart wants A/B/C partitions instead.

OS Stack Bake-Off

After this we had a big OS stack show and tell with people from Debian, GNOME OS, and postmarketOS. From a development/app platform point of view what we need is an OS with:

very recent kernel/hardware enablement (because most phones are still in the process of getting mainlined)
large, active community around the OS, including a security team
image-based, with Flatpak apps
up to date middleware (systemd, flatpak, portals, etc.)
release cycle in sync with GNOME’s
nightly images for testing/dogfooding
aligned with GNOME in terms of design philosophy (upstream first, design first, suitably scared of preferences, etc.)

Currently there isn’t an obvious best option, unfortunately. Debian/Mobian has a large community and recent kernels, but its stable version doesn’t have most of the other things we need since its release cycle is way too slow and out of sync with GNOME’s (and using testing/unstable has other problems). postmarketOS is doing lots of great work on hardware enablement and has a lot of what we want, but the lack of systemd makes it impractical (especially once we implement things like homed encryption). GNOME OS nominally has most of what we want, but the community around it is still too small and not enough people are using it full-time.

We’ll see what the future holds in this regard, but I’m hopeful that one or more options will tick all the boxes in the future!

Shell History and Discussion

Another point on the agenda was longer-term planning for the shell. The Phosh/GNOME Shell question has been unaddressed for years, and we keep investing tons of time and energy into maintaining two shells that implement the exact same design.

Unfortunately we didn’t have enough people from the Phosh side at the hackfest to do any actual planning, but we did discuss the history and current state of both projects. People also discussed next steps on the GNOME Shell mobile side, primarily making it clearer that the branch is maintained even though it’s not fully upstream yet, giving it a proper README, having an issue tracker, and so on.

MPRIS Controls

Oliver was interested in MPRIS buttons, so we had a design session about it. The problem here is that different apps need different buttons in their MPRIS widget depending on the use case (e.g. podcast apps need jump buttons, pause doesn’t make much sense for a live stream, etc.). Currently we have no way for apps to tell the system what buttons it needs, so we discussed how this could be done. The results of our session are written up in this issue, and a vague consensus that this could be done as an extension of the existing MPRIS protocol.

To address the short-term issue of not having jump buttons for podcasts, Jonas statically added them to the mobile shell branch for now.

File Opening

Pablo had encountered some oddities with the file opening pattern in his work on Evince, so we took a look at the question more broadly. Turns out we have a lot of different patterns for this, and almost every app behaves slightly differently! We didn’t get as far as defining if/how this could be standardized, but at least we wrote up the status quo and current use cases in apps in an issue.

Flathub

Bart and Kolja from Flathub also joined on Monday and Tuesday, so we worked on the Flathub website a bit (primarily the app tile layout), and discussed a new initiative to make it possible to show the best apps on the home page. The current idea for this is to start off with a new toolbar on Flathub app pages when you’re logged in as a moderator, that allows a mods to set additional metadata on apps. We’d pair this with an outreach campaign to raise the bar on app metadata, and perhaps automated linting for some of the easier to check things.

Bart and Kolja also worked on finally getting the libappstream port done, so that we can have device support metadata, image captions, etc. from Flathub. They made a lot of good progress but this is still not quite done, because there’s lots of edge cases where the new system doesn’t work with some existing app metadata.

Mobile Platform Roadmap

A few of us sat down and worked on a priority list of things we’d need in GNOME mobile to be able to realistically daily drive it, including platform, shell, and app features. We documented the results in this issue.

Various other things that happened but that I didn’t follow closely:

Pablo and Sonny sat down together and looked over the postmarketOS GNOME session, doing a bit of QA and finding areas where it could be closer to the upstream GNOME vision. Lots of nice improvements since we last looked at it a few months back, and lots more to come :)
There was a session about device wakeup APIs. I didn’t attend it, but it sounded like they made progress towards a portal for this so that e.g. alarms can wake up the device when it’s suspended or powered off.
Robert, Dorota, Caleb, and others had several in-depth sessions on camera hardware enablement, libcamera and the like
Julian got GNOME OS running on the Librem 5 with a recent kernel

GNOME OS running on the Librem 5 for the first time

Overall I found the event very productive and a lot of fun, so thanks everyone for making it a success. In particular, thanks to the GNOME Foundation for sponsoring, Elio Qoshi and Cultivation Space for hosting us, and Sonny for co-organizing. See you next time!

Berlin Mini GUADEC 2022

Given the location of this year’s GUADEC many of us couldn’t make it to the real event (or didn’t want to because of the huge emissions), but since there’s a relatively large local community in Berlin and nearby central Europe, we decided to have a new edition of our satellite event, to watch the talks together remotely.

This year we were quite a few more people than last year (a bit more than 20 overall), so it almost had a real conference character, though the organization was a lot more barebones than a real event of course.

Thanks to Sonny Piers we had c-base as our venue this year, which was very cool. In addition to the space ship interior we also had a nice outside area near the river we could use for COVID-friendly hacking.

We also had a number of local live talks streamed from Berlin. Thanks to the people from c-base for their professional help with the streaming setup!

On Thursday I gave my talk about post-collapse computing, i.e. why we need radical climate action now to prevent a total catastrophe, and failing that, what we could do to make our software more resilient in the face of an ever-worsening crisis.

Unfortunately I ran out of time towards the end so there wasn’t any room for questions/discussion, which is what I’d been looking forward to the most. I’ll write it up in blog post form soon though, so hopefully that can still happen asynchronously.

Hacking outside c-base on the river side

Since Allan, Jakub, and I were there we wanted to use the opportunity to work on some difficult design questions in person, particularly around tiling and window management. We made good progress in some of these areas, and I’m personally very excited about the shape this work is taking.

Because we had a number of app maintainers attending we ended up doing a lot of hallway design reviews and discussions, including about Files, Contacts, Software, Fractal, and Health. Of course, inevitably there were also a lot of the kinds of cross-discipline conversations that can only happen in these in-person settings, and which are often what sets the direction for big things to come.

One area I’m particularly interested in is local-first and better offline support across the stack, both from a resilience and UX point of view. We never quite found our footing in the cloud era (i.e. the past decade) because we’re not really set up to manage server infrastructure, but looking ahead at a local-first future, we’re actually in a much better position.

The Purism gang posing with the Librem 5: Julian, Adrien, and myself

For some more impressions, check out Jakub’s video from the event.

Thanks to everyone for joining, c-base for hosting, the GNOME Foundation for financial support for the event, and hopefully see you all next year!

GNOME Shell Hackfest 2019

This October I attended the GNOME Shell Hackfest 2019 in the Netherlands. It was originally just planned as a small hackfest for core Shell developers, but then us designers decided to crash the party and it became a pretty big thing. In the end we were about 15 people from lots of different companies, including Red Hat, Endless, Purism, and Canonical. The venue was the Revspace hackerspace in Leidschendam, which is somewhere between the Hague and Leiden.

The venue was very cool, with plenty of hackerspace-y gadgets and a room with couches and a whiteboard, which was perfect for the design team’s planning sessions.

Allan, Jakub, and I were primarily there to make progress on some long-standing issues with GNOME Shell, such as new user onboarding, the app grid, and the spatial model of the Shell. We’ve wanted to address many of these things for a long time (in fact, some of them were already discussed at the London hackfest 2 years ago). In the weeks leading up to the hackfest we had already been working on this (together with Sam Hewitt who couldn’t make it to the hackfest unfortunately), preparing a number of concepts to be worked out in more detail.

At the hackfest we made these concepts more concrete, worked on mockups and prototypes, and discussed them with Shell developers. It’s still early days for all of this, but we’re very excited about sharing it more widely soon.

Jakub presenting some exciting prototypes to the Shell developers

We also worked on a number of other things, such as the new lock screen design, which Georges has started to implement, prettier Shell dialogs, and some changes to the system status menu.

Thanks to Carlos Garnacho and Hans de Goede for organizing, Revspace for hosting us, and the GNOME Foundation for sponsoring my travel and accommodation!

Seville Fractal Hackfest Report

Last week I was in Seville, Spain for our second Fractal Hackfest this year. This time it was organized by Daniel Garica Moreno, and held on the University of Seville campus. It was a small event with mostly core developers, focused on driving forward the backend changes and refactoring needed to make our future plans (end-to-end and the app split) possible. We also had some local newcomers join (shoutout to Alejandro Domínguez).

Backend

The main reason why we wanted to have another hackfest was to drive forward the long-term effort of making the application more modular, in order to add persistent storage and allow for the app split. The work done over the past few months (such Julian’s room history refactor) has gotten us much closer to that, but there were still some missing pieces and open questions.

During the hackfest we discussed some of these things, and decided, among other things, to split the backend out into a separate crate and move to SQLite for persistent storage. Daniel and Julian already started working on these things, see their blog posts for more details.

GNOME Newcomer Experience

The other focus for the hackfest was discussing an improved GNOME Newcomer experience. Finding the right rooms to join is currently quite confusing, as there is no central directory of all GNOME rooms. The main way people discover rooms seems to be word of mouth or googling for them, which is clearly not great.

Since one of the main goals for Fractal is to provide a more modern alternative to IRC for GNOME developers, we’ve been thinking about how to make it seamless to discover and join GNOME rooms for some time.

In theory Matrix has communities, which would address use cases like this one. However, since the spec for this is not really settled yet, and implementing it would be a lot of work, we’d like to do something simpler for now.

Looking at some relevant art we found Builder’s integration of Newcomer apps on the project screen quite interesting, because it’s very accessible but doesn’t get in the way.

The idea we came up with is combining the newcomer UX with the room directory into a new discovery view, and moving it to a more prominent location in the sidebar. There are still some open questions about how exactly we’d implement this, but it’s an exciting direction to be working towards.

Mockup for the new first run experience with easy access to both the room directory and GNOME rooms

Hacking, Housekeeping, and Best Practices

We did a lot of overdue housekeeping and organizational work, like moving to GNOME/ on Gitlab (thanks Carlos!), getting a new policy for code reviews and QA in place, issue triage, fixing bugs, and starting the process of documenting best practices we’d like contributors make use of.

Call with johannes about end-to-end encryption. Unfortunately we had to use a University computer running Windows to use the projector :(

New Release!

Lastly, we made a push to fix some of the last outstanding blocker bugs to get a new stable release out the door, which is long overdue (the last release was in September, which is an eternity by our standards). A ton of exciting stuff has landed on master since then: a more legible layout for the message view (using Libhandy’s HdyColumn), smoother message loading, a reorganized header bar with in-window primary menu, and support for large emoji. I’m very excited to finally get these things in the hands of users.

The new release now almost ready, and will be on Flathub very soon. Since there were a lot of big changes under the hood, there will probably be some exciting new bugs as well, so please file issues on Gitlab :)

Thanks to everyone who attended the hackfest, Daniel for organizing, and the University of Seville for hosting us. Thanks also to my employer Purism for sponsoring my travel and accommodation, and the GNOME Foundation, SUGUS, and Plan4D for sponsoring some lunches and dinners. See you next time o/

Announcing the Fractal Hackfest in Seville

It’s been an exciting year for Fractal, the GNOME Matrix client. Since our last hackfest in May, we’ve decided to split the application, refactored large parts of the backend, implemented new features such as the media viewer, made the message history adaptive, and laid the groundwork for end-to-end encryption.

Now that we have most of the foundations in place that will enable our long-term goals (such as adaptive layout, E2E, and the app split), we’re getting together again to push these initiatives forward. This is why we’re having another hackfest on December 11-14 in Seville, Spain.

The main focus of the hackfest will be finalizing the backend refactor, and tying up various loose ends related to this, so we can start working on E2E and the app split. The other area we want to focus on is improving Fractal as a tool for GNOME developers, and IRC replacement. In particular, we’re interested in providing a smooth, integrated GNOME Newcomer experience, because finding the right rooms to join is currently a big pain point for new contributors.

See you in Seville!

Banquets and Barbecues

tl;dr: We’re splitting up Fractal into two separate apps: One to replace IRC, the other to replace Telegram.

This is an in-depth post on the thinking behind the split of the Fractal app, which was decided at the hackfest in Strasbourg last week. For more information about the hackfest, have a look at my other blog post.

1-1 woes

One of the biggest problems with Fractal at the moment is that 1-1 messaging is pretty terrible. Since the rooms in the sidebar are sorted by most recent activity, high-traffic public rooms (such as GNOME IRC channels) tend to drown out rooms with less traffic, such as 1-1s and small groups. This is problematic because the signal-to-noise ratio in 1-1 chats and small groups tends to be much higher than in high-traffic public rooms. This leaves the user constantly searching for the rooms they care about, while the rooms they don’t care about are always at the top.

1-1 chats are quickly drowned out by high-traffic public rooms in the sidebar

One way to solve this problem is having a favorites group for “important” rooms. This is a feature Fractal has had for a while, and it does solve some of the problems with a room list sorted purely by recent activity. However, it only works well for rooms that are important over long periods of time, and needs to be managed manually. 1-1 chats are often brief, and there can be many of them in parallel. Putting them in favorites doesn’t make sense in many cases, as it would balloon the size of the favorites group, and require lots of manual work when starting or ending a conversation.

The “obvious” solution would be doing what Riot does: Having a separate group of 1-1 rooms in the sidebar, and thereby keeping the 1-1 conversations in one consistent place. However, this creates more problems than it solves. In practice, it results in multiple groups of arbitrary length competing for real estate in the sidebar. If you have a lot of 1-1s, this means that you’ll be able to see very few rooms (even when most of the 1-1s are old and not relevant at the moment). In Riot, this group is capped at 10 visible rooms by default, but that’s still not great if you only need 2 of them at the moment. The category can be collapsed, but then you can’t see which 1-1s have new messages, and it also means lots of busywork collapsing/expanding the group. Clearly this isn’t an ideal solution, which is why we were very hesitant to go down this path.

Riot’s separate 1-1 category doesn’t really solve the problem, because old 1-1s take up a ton of vertical real estate when it’s expanded

A way out?

As we were discussing this issue over the past few months, I started looking more closely at the way people use different messaging tools. One thing I found puzzling is that despite the fact that Matrix theoretically supports the use cases covered by popular apps like Whatsapp and Telegram, few people are actually using it to replace those apps. Instead, they use it to replace IRC and Slack.

Why? My theory is that most chat rooms fall in one of three categories:
Private Chats, which include 1-1s and small groups; Team Chats, which are larger, but still private and invite-only; and Public Rooms, which are basically like IRC.

Team Chats and Public Rooms share many characteristics: Both have relatively high amounts of traffic, and there’s a lot of noise. The main difference is that Team Chats are private and the members rarely change (e.g. a company’s internal Slack), while Public Rooms can be joined by anyone at any time, and there is no expectation of privacy (e.g. #gnome-hackers on IRC).

However, Private Chats have relatively little in common with the other two categories: They are low-traffic, and have little or no noise. This may sound like a small difference, but I think it’s the reason why 1-1s suck in Fractal/Riot/IRC, and why people aren’t using Matrix to replace Telegram.

The Banquet and the Barbecue

I’ve come to the conclusion that one app can’t cover all the use cases that the Matrix protocol supports, and still provide competitive UX. If you design an app to deal with lots of high-traffic rooms (e.g. Riot as it is today), it will suck for 1-1s, so people will use something else for those. Similarly, Telegram is primarily designed for 1-1s and small groups, which is why it’s a terrible experience if you have many high-traffic groups.

If we want Matrix to succeed as more than an IRC/Slack replacement we need multiple apps, each focusing on a distinct use case. For messaging, I think the most important distinction to make is between what I call the Banquet and the Barbecue.

Slack is one of the most widely used apps covering the Banquet use case

The Banquet is a big, loud place. There are tons of people, and you don’t know many of them. Lots of things are happening all the time, and it’s hard to keep track of everything. This is what Matrix is currently mostly used for. Slack, IRC, and Discord are also all in this category.

iMessage is a good example of an app focused on the Barbecue use case

The Barbecue is at the other end of the spectrum: It’s a calm, private environment where friends, family, co-workers, and other acquaintances hang out. Conversations are mostly between 2 or 3 people, slow, and often very personal. Telegram, Whatsapp, iMessage, Facebook Messenger, and a myriad of other chat apps are optimized for this use case.

Fracturing Fractal

Now, what does this mean for Fractal? After a long discussion on Thursday, we decided to split up Fractal into two separate apps with different interfaces, each containing a subset of the user’s Matrix rooms.

Exactly how rooms will be split between the two apps is not 100% clear yet. 1-1s are clearly Barbecue, public rooms are clearly Banquet, but private groups could go either way. For these cases we may need a way to explicitly move rooms between apps. The distinction should probably be part of the Matrix spec, so the intent for a room to be a Barbecue or Banquet room could be set when creating a room, and persist across devices.

The two apps will share practically all the internals, and even large parts of the interface. However, the split will allow us to do some things differently in each app to optimize the interfaces for the different use cases. Some of the changes we’re considering are a bubble-style message view in the Barbecue app, and more room categories (such as low-priority) in the Banquet app’s sidebar.

For more details on the split have a look at the blog posts by Daniel, Eisha, Julian, and Adrien.

Messages and Discussions

How exactly the apps will be branded (and what will happen to the Fractal name we all love) is still being decided, but there is some consensus to move to GNOME-style generic names. The Barbecue app will almost certainly be called “Messages”. For the Banquet app there’s less agreement, but my current favorite is “Discussions”.

Early-stage mockups showing what the two different apps could look like

The Fractal brand will not go away though: We’re thinking of keeping it around as the name of the community project that develops both GNOME Matrix apps, and/or using it for the backend powering both apps.

There are lots of details to be figured out in this transition, both from a design and an implementation perspective, but I’m very excited about this new direction. If you’d like to join the effort, come talk to us on Matrix.

Note: I have no illusions that this change will magically get everyone to leave Whatsapp/Telegram/iMessage and move to Matrix. In the short term, the goal is simply to make Matrix 1-1s a good experience. That said, if we ever want Matrix to make inroads with the general public, I think a move in this direction is an important precondition.

Fractal Hackfest in Strasbourg

Last week we had an intense 4-day hackfest in Strasbourg to map out the future of Fractal, a native GNOME Matrix messaging app. The event was held at Epitech in Strasbourg’s old town, and organized by Alexandre Franke. Among the attendees were core Fractal contributors Daniel, Alexandre, Eisha, and Julian, as well as Dorota, Adrien, and Francois from Purism. Special thanks go to Matthew from the Matrix core team for joining us on the first two days.

Our main priorities for the hackfest were to plan the roadmap for the next months, decide on the tasks for our GSoC students (Eisha and Julian), and work on the design of some important missing features, like the room settings.

I personally attended the hackfest in both my role as designer on the Fractal project and as a Purism employee currently working on the apps for the Librem 5. One of the reasons why several members of the Librem 5 team attended the hackfest was that we will need a Matrix messaging app on the phone and wanted to explore a potential collaboration.

The hackfest was extremely productive, so much so that I’ll need multiple blog posts to report on all the things we worked on. Here’s a quick outline of some of the most important things that happened:

We’re splitting the app into two separate apps (more on this in a future blog post)
A big refactor of the backend is happening soon to enable the split
We discussed having a system-level Matrix daemon, which different apps could use as a backend (e.g. the two different messaging apps and a calls app)
Matthew explained that room types will be simplified into 1-1, private groups and public groups in the future (which nicely complements our split). We discussed whether 1-1s should be immutable (they should :P)
Matthew explained how end-to-end encryption and calls work in Matrix, and how we could get them in Fractal
We came up with an initial design for multi-account (which basically consists of an account chooser at startup, and a separate window for each account)
We discussed a design for read receipts. Not quite done yet, but we’re on the right path, I think.
We talked about what it will take to make Fractal work on mobile. Not too big of a problem design-wise, but we’ll need Rust bindings for libhandy and emeus
Eisha will be investigating i18n, because we really want to make the app translatable (currently this is hard to do because we use Rust)
Julian will be working on a big message history UI refactor/redesign, as well as other UI stuff, such as user account settings
There is a huge number of message types we don’t support yet and we discussed the design for most of them (including in-app viewers and a history of sent files)
With Matthew’s help, Daniel fixed the slow initial sync
Julian worked on a nicer UI for the GTK emoji chooser and Rust bindings for gspell
Alexandre landed the new, prettier default avatars
Quentin started packaging Fractal for macOS
We fought our way though the garbage fire that is the Join and History settings in Riot, and emerged with a design that isn’t terrible (thanks Dorota and Julian!)

Matthew explaining device verification for E2E chats

Thanks everyone for attending, Epitech and Alexandre for the venue, and Purism for sponsoring the hackfest! It was amazingly productive to have everyone in one place, and I look forward to seeing you all at GUADEC in July :)

Announcing the 2018 Fractal Hackfest

tl;dr: Fractal is a new Matrix app for GNOME. Come join our hackfest May 10-13th in Strasbourg!

For the past few months, I’ve been contributing to a new group messaging app called Fractal. Its aim is to be so good that we can maybe, eventually, finally replace IRC as the primary communication channel for GNOME development.

IRC is old

Though IRC has been around forever, it’s hardly a good user experience by today’s standards, and can be a significant hurdle to onboarding new contributors (especially non-technical ones). Some of the main issues are:

No easy way to use it from mobile
You can only receive messages while online
Sending images or files is only possible via URL

This has led to some free software projects going so far as to use proprietary services such as Slack. However, even then you’re stuck with an Electron app, which is slow and not integrated with the desktop at all.

Enter the Matrix

The Matrix protocol is a modern alternative to IRC. It provides features such as persistence, inline media, and multi-device syncing. Importantly for GNOME, it is also backwards-compatible to IRC using bridges, allowing us to use the fancy new Matrix features without being cut off from the rest of the community.

Over the past few years, some GNOME developers have started using Matrix, but the lack of a good native client has held back adoption.

Riot in Revolt — The Riot web app works, but it’s not exactly a clean, focused experience

And that’s where Fractal comes in: It’s a native Matrix client for GNOME, with an interface focused on the IRC use case (discussions in large, high-volume rooms). We want it to be a fast, native alternative to Slack that can compete on UX, and doesn’t require non-free network services.

Fractal

I started contributing in late October, when the app was still called Guillotine, and looked like this:

I got involved after Daniel Garcia Moreno, the app’s primary developer, mentioned that he was looking for a designer to help out with his Matrix app on #gnome-design. We then started working on revamping the somewhat haphazardly designed interface, and making it look like a proper GNOME app.

Over the past few months the community has grown from just Daniel and I to over 120 people in our Matrix room, and a number of regular contributors. We’ve redesigned every part of the app (except for the room directory), and added a number of small but essential features, such as invites, room creation, and username autocomplete.

The app now looks like this (though the design is still evolving quite rapidly):

Though it’s still a bit rough, Fractal is already very usable as a day-to-day Matrix client. You can install the Flatpak from Flathub and try it yourself. Bug reports are always appreciated.

The Hackfest

So where do we go from here?

For the past months we’ve mainly been working on overhauling the interface, but we haven’t added many new features. As most of the foundational interface work is done now, we can start planning larger new initiatives.

That’s why next month (May 10th-13th) we’re going to have a Fractal Hackfest in Strasbourg. The core contributors will meet for the first time and discuss exciting new things, such as a better onboarding flow, a new responsive layout, and support for more Matrix features.

Among the people joining the hackfest will be Matthew from Matrix, and some of my coworkers from Purism. It will be a great opportunity for plotting future collaborations.

If you’re interested in joining the hackfest, add yourself to the hackfest wiki page or come talk to us in the Fractal Matrix room.

UX Hackfest London

Last week I took part in the GNOME Shell UX Hackfest in London, along with other designers and developers from GNOME and adjacent communities such as Endless, Pop!, and elementary. We talked about big, fundamental things, like app launching and the lock/login screen, as well as some smaller items, like the first-run experience and legacy window decorations.

I won’t recap everything in detail, because Cassidy from System76 has already done a great job at that. Instead, I want to highlight some of the things I found most interesting.

Spatial model

One of my main interests for this hackfest was to push for better animations and making better use of the spatial dimension in GNOME Shell. If you’ve seen my GUADEC Talk, you know about my grand plan to introduce semantic animations across all of GNOME, and the Shell is obviously no exception. I’m happy to report that we made good progress towards a clear, unified spatial model for GNOME Shell last week.

Everything we came up with are very early stage concepts at this point, but I’m especially excited about the possibility of having the login/unlock screen be part of the same space as the rest of the system, and making the transition between these fluid and semantic.

Tiling

Another utopian dream of mine is a tiling-first desktop. I’ve long felt that overlapping windows are not the best way to do multitasking on screens, and tiling is something I’m very interested in exploring as an alternative. Tiling window managers have long done this, but their UX is usually subpar. However, some text editors like Atom have pretty nice graphical implementations of tiling window managers nowadays, and I feel like this approach might be scalable enough to cover most OS-level use cases as well (perhaps with something like a picture-in-picture mode for certain use cases).

We touched on this topic at various points during this hackfest, especially in relation to the resizable half-tiling introduced in 3.26, and the coming quarter-tiling. However, our current tech stack and the design of most apps are not well suited to a tiling-first approach, so this is unlikely ot happen anytime soon. That said, I want to keep exploring alternatives to free-floating, overlapping windows, and will report on my progress here.

Header bars everywhere

A topic we only briefly touched on, but which I care about a lot, was legacy window decorations (aka title bars). Even though header bars have been around for a while, there are still a lot of apps we all rely on with ugly, space-eating bars at the top (Inkscape, I’m looking at you).

Screenshot of a full-screen Blender window with a title bar — On a 1366x768px display, a 35px title bar takes up close to 5% of the entire screen.

We discussed possible solutions such as conditionally hiding title bars in certain situations, but finally decided that the best course of action is to work with apps upstream to add support for header bars. Firefox and Chromium are currently in the process of implementing this, and we want to encourage other third-party apps to do the same.

Screenshot of Firefox with client-side decorations — Firefox with client-side decorations (in development)

This will be a long and difficult process, but it will result in better apps for everyone, instead of hacky partial solutions. The work on this has just begun, and I’ll blog more about it as this initiative develops.

In summary, I think the hackfest set a clear direction for the future of GNOME Shell, and one that I’m excited to work towards. I’d like to thank the GNOME Foundation for sponsoring my attendance, Allan and Mario for organizing the hackfest, and everyone who attended for being there, and being awesome! Until next time!