An other year, an other FOSDEM

I have come a long way since my first time at FOSDEM a couple of years ago. The first time it was all new and unknown. I tried to attend as many talks as possible, but could only see half of the talks i wanted to go (amazing how many people there are at FOSDEM). Every year I listened to fewer and fewer talks, because conversations I had outside of talks are so much more fun and appealing. I think the biggest thing which changed is that I’m no longer just a user of free software, but an active contributor.

This year, I spent a lot of the time at the GNOME booth, which is always fun. The GNOME beers event is also awesome, though it was really crowded this year (let’s hope we get a bigger space next year). On Friday we also had a great lunch at a Libanese restaurant. Thanks to Adrien for organizing, and thanks to Purism for offering lunch.

While there, I also did some work on Fractal. I investigated why Fractal uses a lot of CPU and memory and made changes to reduce memory usage.

I stayed at a flat together with Bastian, Niclas, Florian, and Tobias. Thanks a lot for the good time in Brussels, especially to Bastian for organizing the apartment. Special thanks to the GNOME foundation for sponsoring the trip to Brussels.

Fractal Hackfest in Seville

Last month I was in Seville for the second Fractal Hackfest. It was a bit of a different experience for me than the other hackfests I have been too, as I’m a core member to the project now, and I also knew most of the other people already.

My main focus for this hackfest was to push forward the work on the Fractal backend. The backend should handle persistent storage and the preparation of data needed in the UI. Right now many things which should conceptually be in the backend are in the frontend, and can therefore cause UI freezes. Over the past months I have been working hard on refactoring the code so we can just drop in the backend without making many changes to the UI code. So the core of the refactors I did was to make sure that data flows in one direction and we don’t keep different copies of the data around.

Backend

For storage we decided to use SQLite instead of LMDB because we have many relations between the data. The data will be structured into 3 tables: USERS, ROOMS, MESSAGES. This gives us a lot of space to expand and allows us to reference to other data with ease.

The backend will have use a glib based API to communicate with the frontend, and on the other hand it will have a Rust only interface to communicate with fractal-matrix-api.

Community

Lunch at a spanish Pizza place

We also had really awesome community bonding events, and some local newcomers joining the hackfest. Thanks to Daniel for organizing the event and also all locals who helped him.

I fell in love with GAction

A few weeks ago, while working on Fractal, I rediscovered something I had completely forgotten about: GAction. I don’t remember how I came across it, but it was definitely love at second sight, because for the next days I didn’t do much other than refactor Fractal and replace code with Actions wherever I could.

Of course, if you like something a lot you have to be careful to use it only where it’s appropriate. For me it was basically when it’s a user action then it should use GAction, or in other word if it’s a response to an user input, like clicking on a button, then it should be a GAction.

Now, how do you use GAction? Since GAction is only an interface we need to use a class implementing it. The most simple class we can use is of course GSimpleAction as the name already implies. Since I’m writing most of my code in Rust these days, all examples will be in Rust.

// We have to create a SimpleActionGroup
let actions = SimpleActionGroup::new();
// Currently we can't use any Variant type we want, we are limited to the most basic ones like strings, numbers, and bools
let reply = SimpleAction::new("reply", glib::VariantTy::new("s").ok());
// And then we have to add it to the ActionGroup
actions.add_action(&reply);
// The last thing we have to do is to connect to the activate singal
reply.connect_activate(move |action, data| {
//Do whatever you want in here, e.g. open a file dialog
});

The above code shows how an action is created. Let’s see how you can use it.

The first thing we need to do is add the created ActionGroup to a Widget. The action will then be available for all children of that widget, so it’s best to add it to a Container or a Box which allows us to have children.

// Create a Box
let b = gtk::Box::new(gtk::Orientation::Horizontal, 0);
// Add the action group we created before to the widget
box.insert_action_group("something", &actions);

Now the actions are available to all children of the box with the prefix “something”. In our example “something.reply” would be the name of the action. Given a GTKButton you can set the action_name and the target_value of the widget, because it implements GtkActionable. The code basically looks like this:

// Create a new GTK button
let button: gtk::Button = gtk::Button::new();
// We need to use a Variant to store the string we want to set as the target value
let data = glib::Variant::from("some string");
// Set the action name
button.set_action_name("something.reply");
// We also need to set the target value
// If we don't do this, GLib will complain that we didn't call the action with the correct target and won't execute the action
button.set_action_target_value(&data);

You could also set the properties via the .ui file. When you replace the event handlers of clicks with actions you can really streamline your code base.

I showed the basic usage of the GAction in the above examples, but you should definitely have a look at the GAction documentation to see all the potential it has. Also, GPropertyAction is awesome because it allows you to control GObject properties directly via Actions. Sadly, the Rust bindings for it are not in a stable release yet, but they were added to master a couple of months ago.

Better Room History in Fractal

Over past month I’ve been sponsored by Purism to work on improving the message view in Fractal. This post will highlight the biggest and most interesting changes.

The first thing I improved was how older messages are added to the message view when scrolling back. Before, there was a jarring cut when new messages were loaded, but now you can just scroll upward and older messages are loaded continuously. This makes it much easier to search for a message in the history, because there are no sudden jumps when messages are added to the list. In the video you can see the how the smooth history loading works.

The part I’m most excited about is the new “new message divider”. Here’s a small screen cast to show the new behavior:

When the user opens a room they can directly start reading the conversation from the last seen message and they don’t need to search for the new message divider. Not only has the UX gotten a lot better, but also the underlying code is much cleaner now. Sadly, it doesn’t work (for now) if Fractal doesn’t have a “last seen mark” for the room. This can happen, for example, when there are more than 40 not seen messages. The reason for that is that Fractal only syncs the last 40 messages of a conversation. The new message divider also disappears after a couple of seconds and this time is not yet connected to how many messages an user has to read.

I also spent some time on making message rendering faster. I replaced the RegExp with more efficient code, this made the rendering much faster (from ~10ms to ~ 1ms) for every single message. This is already pretty good, but I think there’s still room for improvement.

In summary, all of these things improve Fractal’s UX a lot and make it feel more like a modern messaging app. To further improve the UX we need to also update the app’s backend and improve how we get data from the Matrix server. Thanks to Purism for sponsoring this work, Daniel for reviewing and merging everything, and everyone else who gave feedback and helped test this.

We don’t have a new stable release with these changes yet, but they have been merged, and you can already try them on Master.

Rust ❤️ GNOME Hackfest

Last week I was in Thessaloniki, Greece for the GNOME+Rust Hackfest #4. I liked the city, but sadly during the weekend we had really bad weather, and it was much colder then I excepted. The hackfest itself was awesome though. The thing I liked the most was starting to contribute to new projects. I guess it’s so much easier to get started when you have the maintainer sitting next to you and you can bother them with your silly questions ;).

I had some time to do sightseeing as well. The city has a long and interesting history. Everybody who goes to Thessaloniki should go to one of the most known monuments of the city: the white tower. It gives you a pretty good overview of the history of the city. And also the panorama view you get on top of the building is really good.

At the beginning of the hackfest I was kind a shy and worked mostly on Fractal, but during the next days I really wormed up and started contributing to the GTK Rust bindings and librsvg.

It was a lot of fun and I’m looking really forward to next year’s GUADEC, and to go back to Thessaloniki. Sebastian and Vivia, you did a awesome job at organizing this hackfest, thank you so much!

PS: the food was really good ;)

Purism Fractal sponsorship

I’m happy to announce that Purism agreed to sponsor my work on Fractal for the next couple of weeks. I will polish the room history and drastically improve the UX/UI around scrolling, loading messages etc. which will make Fractal feel much nicer. As part of this I will also clean up and refactor the current code. On my agenda is the following:

Smooth history loading

Loading old messages in the history is currently a bit jarring, because the scroll position isn’t preserved when new messages come in. I’d like to address this by loading messages outside the viewport, making it so that the user isn’t even aware that more messages are being loaded most of the time. This is a crucial part of why modern messaging apps feel so nice.

Faster message rendering

There are some inefficiencies in how messages are currently rendered, which make showing messages not as smooth as it could be. Fixing this could improve the experience of sending/receiving messages significantly.

 “New messages” behavior

  • Re-add a “New messages” divider, since it was lost as part of the big history refactor I recently completed
  • Scroll to last seen message when opening app instead of most recent message
  • Fix bugs in current behavior and make sure the divider always shows up

Add day label

Add a label with the day/date at the beginning of every new day, like other messaging apps do.

The end of GSoC

After three months of hard work and a lot of coding the Google Summer of Code is over. I learned a lot and had a lot fun. GSoC was an amazing experience and I encourage everybody to participate in future editions. At this point I’ve been a contributor to GNOME for nearly a year, and I plan on sticking around for a long time. I really hope that other GSoC students also found it so enjoyable, and keep contributing to GNOME or other Free Software Projects.

This will be my final GSoC post, so I will summarize what I did for Fractal in the last couple of months, and what I didn’t finish before the deadline. I completed some of the tasks I proposed for GSoC already before the Coding Phase. It was really nice that we had a meeting with Daniel (my mentor) and Eisha (my co-GSoC student) every week, and also some other core members working for Fractal joined sometimes. During these meetings I could get feedback from Daniel and discuss tasks for the next week. We had a really good workflow. I guess that was also a reason that I started working on some imported tasks and not only on things I initially proposed.

Main Tasks

These are my main Tasks I completed during the GSoC. They are all already merged, and included in the current release:

These are the task I could not finish in time:

  • Spell Check: Add spellcheck to Fractal, based on gspell. I implemented this (though not with the UI we’ll eventually want) and it was merged, but since then we made some big changes to the message input widget and therefore it doesn’t work anymore at the moment. We also decided that other tasks were more important. I will work on this in the next couple of months.
  • refactor room history: A big change to how the message history is displayed, loaded, and stored. This will lay the groundwork for a lot of new features we want, and the backend refactor we need to split the app. This is one of the biggest tasks I worked on during the summer, and I will try to finish it as soon as possible. Since we started restructuring the code base I had a lot of stuff to figure out and therefore it was delayed.
  • Use libhandy to have adaptive columns: We want Fractal to work on big screens as well as on small screens and therefore we use a widget in libhandy to center columns and limit the width of, mostly, gtkListboxes.

During the Summer I made also some other contributions, which were tangentially related to Fractal and my GSoC tasks, e.g. I fixed the GTK emoji picker layout and styling (this has been merged now, so look forward to a much nicer emoji picker in GTK 3.24).

My Code

Most of my work is upstream and already integrated into Fractal. This is a list of all commits merged into Master:

* b9d7a5a - message_menu: remove appOP dependecy from message_menu 
* 6897ad0 - roomsettings: update .po file and add i18n 
* f96eeef - roomsettings: fix member list disappearing when closing 
* c772d9f - roomsettings: hide confirm button when the topic/name didn"t change 
* cf7dffd - roomsettings: clean up code from old memberslist 
* 44b2d28 - roomsettings: reload room settings when members are avaible 
* 089b001 - headerbar: remove avatar form the headerbar 
* 20c9ebd - roomsettings: move room settings panel to a custom widget 
* 692045f - roomsettings: make room description label a dim-label 
* 78c5be1 - roomsettings: set max with for centerd column 
* 0c99e0e - roomheaderbar: move room menu to new settings button 
* a2de6ac - roomsettings: hide not implemented widgets 
* 5097931 - roomsettings: request avatar when entering the room settings 
* 0244a23 - roomsettings: hide settings not needed for a room type 
* dc83201 - roomsettings: add invite button 
* 544fc52 - roomsettings: show members in the room settings 
* 4e6b802 - room: add room settings panel 
* 8993f5f - avatar: refactor avatar loading and caching 
* b81f1ec - accountsettings: add closures to address button callback 
* 200440d - accountsettings: remove seperator in user settings menu 
* e88663e - accountsettings: make animation for destruction revealer slower 
* 0068fde - accountsettings: fix valignment of label name/email/phone 
* 140da51 - accountsettings: scroll to advanced/destruction section when open 
* f1e86bd - accountsettings: add placeholder text 
* c4d608b - accountsettings: show spinner will updating the avatar 
* 41a07a8 - accountsettings: generate client secret for each request 
* 7c459d3 - accountsettings: use stored identity server 
* 708c5c0 - accountsettings: add button to submit new display name 
* e019e48 - accountsettings: move account settings to the main window 
* 6ee5c99 - accountsettings: show confirm dialog when deleting account 
* 9a2dada - accountsettings: remove loading spinner once password is set 
* 835ab15 - accountsettings: password change and account destruction 
* ba74610 - accountsettings: managment for phone and email threePIDs 
* 345cb8e - accountsettings: add buttons to address entries 
* 04ed27f - accountsettings: load threePIDs (email and phone addresses) 
* a214504 - accountsettings: password validation 
* 8711c01 - accountsettings: add password change dialog 
* 1d31eab - accountsettings: add api for changing account avatar 
* de83098 - accountsettings: add api for changing account name 
* 96bcb94 - accountsettings: add UI for account settings 
* 1fec2e5 - login: use global constant for homeserver and identity server 
* ee1ad9c - login: store identity server 
* 206035c - gspell: add basic spell check 
* f92387c - mention: highlight own username in mentions using pango attributes 
* 3d3a873 - fix: add joining members to the there own room instate to the active room 
* d6145e8 - message-history: set space after last message to 18px 
* 972d44a - mention: blue highlight for messages with mentions and cleanup css file 
* 0571245 - autoscroll: add ease-out when scrolling to last message 
* b35dec6 - autoscroll: add button to scroll down to last message 
* 789048f - autoscroll: move to new message when the user send a message 
* 55dcaf1 - autoscroll: no delay before autoscroll 
* 77051f6 - center inapp notification 
* 5c1cd22 - fix autoscroll, disable autoscroll when not at the end of message history, fix #137 
* f59c3d2 - autocomplete: fix change usernames font color to white when selection ends at the same position as the username 
* f6a3088 - move autocomplete popover to a seperate file 
* 0f2d8a8 - messages-history: add spacing after last message 
* b548fc9 - messages-history: remove blue selection and add more space inside each row 
* dc0f1e1 - add Julian Sparber to the author list 
* 20beb87 -add spacing bethween avatar and msg body 
* b721c96 - add padding to the message history 
* 533a736 - implement redesin of the autocomplete popover, fix #146 
* 9a9f215 - remove " (IRC)" from the end of suggested username for mentions, fix #126 
* 101c7cd - use only @ with tab-compleation 
* 798c6d3 - match also uid for mentions 
* 3caf5a0 - limit user name suggests to 5 and allow usage of @ for mention 
* b2c5a17 - fix spacing in user menu popover and in room settings popover 
* 8fa9b39 - reoder options in add room menu, add separator and change some lables 
* 5f15909 - fix title of dialog for joining a room by ID 
* b475b98 - center all dialog title on dialog width and refactor glade file 
* d9059f0 - fix the text of some lables 
* e5827c5 - spawn popover from room name and remove room-menu-button 
* 7b72f59 - add image to no room view and add new style #132 
* e92b20a - fix alignment of no room selected message by splitting the text to two label 
* 6f2fa3b - fix alignment for the text when no room is selected 
* fcb8b41 - set focus for each stack view, fix #118 
* c2a2816 - increase avatar size in the sidebar to 24px 
* d7dc175 - make user menu button avatar 24px 
* f94c558 - make room avatar in header 24px if no description 
* 7b4fe55 - show user info in popover 
* 9a0d36b - split user menu in two menus and remove the title in the left headerbar 
* 3c80382 - add spinner to initial sync notification 
* 1db7847 - [refactor] use headerbars in the titlebar instate of boxes 
* 2538993 - adjust spacing on first message in group 
* 6db14b6 - set timestamp fontsize to small 
* 3a92de6 - fix spacing around messages 
* f355fdc - add spacing to load more button and center it 

Some of my work went into libraries so it can be used by other applications, and one big merge request is still work in progress. I will continue to work on these git repositories, therefore I added the last commit for each repo done during GSoC:

The Gspell rust bindings:
https://gitlab.gnome.org/jsparber/gspell-rs
(last commit: 1501c997e2450378ea7e8291c94fa8189bb360df)
https://gitlab.gnome.org/jsparber/gspell-sys-rs
(last commit: 158e8ebbe5aeafc96663e3e76fc9a25715198264)

The libhandy rust bindings:
https://gitlab.gnome.org/jsparber/libhandy-rs
(last commit: f174e76882896c32959d0d16fe11eaadfce1c674)
https://gitlab.gnome.org/jsparber/libhandy-sys-rs
(last commit: 2caa9f6e0b68d391a6cdb45123f85c73b9852553)

I also created a libhandy test branch in Fractal, but I got never around merging it, because of a bug in libhandy (though it is fixed now, so it could be merged soon)
https://gitlab.gnome.org/jsparber/fractal/tree/handy
(last commit: b28da9a7443c34989dc8703402e6bc154a962f57)

The code to generate an avatar based on usernames:
https://gitlab.gnome.org/jsparber/letter-avatar
(last commit: 083e65e6ae8f63dc4071d3fac47adaa81e99d607)

There’s already a work in progress MR for the room history refactor:
https://gitlab.gnome.org/World/fractal/merge_requests/184
(last commit: 38abd3ceb3eb6d0ef1e70f66c7b0139ad65413b4)

GUADEC 2018

From the 2th of July I have been travelling from Italy all the way to the south Spain by train, to attend GUADEC 2018. During this long trip, I didn’t just sleep, but I kept working on Fractal and some other cool things.

The biggest thing I worked on was the refactor of the code which handles avatar download and display in Fractal. I removed a lot of code because we had more than 3 different methods to load an avatar. Then I replaced the different methods with an easy to use function, which also handles the fallback when there is no avatar and it needs to be generated from the user’s name. I found that the fractal-api (which should only make http requests to the Matrix homeserver) had a dependency to GTK+. The code which was in the fractal-api to generate a fallback avatar using the username is now available for other projects on crates.

Hackfest on the train 😍

I was travelling with Tobias, and at some point we randomly met Bastian on the train from Madrid to Almera, so the travel turned into a hackfest on the train.

GUADEC

This year was my second GUADEC, so it was really exciting to see all people again I met last year. The most awesome thing this year was that many people where not strangers anymore, I knew them already from last year, from IRC or from somewhere else online. The last year was a really exciting year, because I got way more involve in GNOME than I’d ever dreamed of. This year I also volunteered to help out at the info/registration desk which was a lot of fun.

I also gave a 3 min talk about my work as a GSoC student for Fractal. I also met all the other GSoC students and it was really fun to see so many young people working on GNOME, I really hope most of them will stick around also after GSoC.

Icon exporter

A few months ago, I created a small shell script to export slides form a svg file, but it uses Inkscape and it is terribly slow. During the conference days I started playing around with librsvg to export slides from a svg file, and learned a lot about librsvg. I can’t remember how the idea came up but at some point I was suddenly working on a rust app to export the Adwaita icons from the stencils svg file. This new approach looks really promising, as the new app exports all the symbolic icons in just few seconds. However, right now there are some icons which are not generated correctly, most likely because of rounding errors. I will investigate more and maybe make my first MR to librsvg. I wear a hoodie on a lot of pictures, because it was really cold in some of the rooms, and I can’t stand when the AC is directly over my head.

Going back to Italy

Tobias and I took also the train to go back home, but we stopped in Granada and Madrid for a couple of days. During the trip home I finished the new room settings layout for Fractal, which was a big piece of work because I tried a couple of new things, which soon will be applied to other parts of Fractal, in order to make the backend split happen (which will in turn enable splitting the app into two separate apps).

So much fun

I had so much fun during all the talks, social events, and BOFs. Thank you to all people who came to GUADEC 2018 and a even bigger thank you to all the amazing people who made it happen. I’m already looking forward to next year. I will leave you with picture of the sandcastle BOF at the beach. 😉


 

Refactor: Backend and UI

Fractal is currently structured into two parts: The API part (fractal-matrix-api) and GTK part (fractal-gtk). The first one mostly just does the https calls to the Matrix server, the GTK part does everything else. This post will not talk about the API part since that will remain more or less the same (at least for now).

Why do this split?

Our goal is to create two separate Apps for two different use cases: Banquets and Barbecues. For this split to happen we first need to make some changes to Fractal’s architecture.

I think to keep Fractal maintainable in the future we need to simplify especially the AppOp struct, since it has been growing at an alarming pace (Don’t get me wrong I’m really happy that Fractal improves so quickly). AppOp is the struct which contains all widget references and most methods. Basically, it contains everything important.

Also, a lot of Matrix related code is intertwined with GTK code, and therefore many tasks are handled in the main thread.

New architecture

The communication between fractal-matrix-api and the Fractal backend will stay the same. Therefore, the backend-loop will live in the backend, but instead of making UI changes directly, it will just store the new data, and then signal the main GTK application that it has to update the related UI component. The backend will expose a set of methods to give the main application access to the data and to change them.

The main application will create all UI components and keep them up-to-date. At last we have the UI components which are “custom widgets”, such as the sidebar. A custom widgets has two public methods, one to create a new component, and another one to update its content.

When the main application receives a UI update event, containing only information about which UI component should be updated, but no other data, the application uses the interface offered by the backend to get all needed information to call the update method of each custom widget. The update method then decides then which GTK widgets have to be destroyed, and which can be reused to display the new data.

How to get started?

The first thing we can do is switch form Mutex to RwLock, this will already solve many UI freezes since we don’t need to look the data anymore to read it.

Then we can start moving the current code form AppOp, where possible, to a custom widget (the Rust structs we are using right now). Each widget should have a create and an update method, for creating the new component and for keeping it updated without the need of destroying it first. (Address.rs is a widget example created this way). Most custom widgets already have the create method called widget() (if I remember it correctly) but they are missing the update method.

Nothing is set in stone

We didn’t decide anything yet. I wrote this blog post to start the discussion about how to tackle the refactor and split, so input/comments/feedback are very welcome!

Fractal GSoC Progress

In the last two weeks I have been working on the user account settings for Fractal. I can’t wait to get to the point where Fractal will work stand-alone without needing to use Riot for certain things, and getting complete account settings is a major step in that direction.

Let’s start with the status of Fractal before I started my task. The only thing Fractal did was loading the display name and the avatar, and they couldn’t even be changed.

Design

Tobias had made mockups for the account settings last fall, but they did not contain all the necessary features and was never fully implemented. During the Strasbourg hackfest we started discussing a more complete design. Since then we’ve been expanding on that work, which led us to the current design:

Coding

I started creating the UI file which describes the view. I tried to focus just on the UI, in order to not overcomplicate my process. The first thing was just a dialog with all widgets without any behavior.
The second step was to add all information already available in Fractal, like homeserver, MXID, avatar, and display name.

Until this point the implementation was quite easy because it was mostly GTK code, which I’m quite familiar with.
However, up until two weeks ago I had never touched the fractal-api code, so the rest was trickier. Most APIs, e.g. setting the avatar, are quite easy to implement. The most complicated are the calls made for handling third party identifiers (https://matrix.org/docs/spec/client_server/r0.3.0.html#adding-account-administrative-contact-information) (email addresses and phone numbers). It is not only about setting the new identifier, but the client has also the request a token to make sure the user is the owner of the email address.

Current State

The user can now change the following settings: Avatar, display name, add and remove email addresses and phone numbers. Also, they can see the homeserver and their own MXID.
I already added all the API calls except for the calls needed for account destruction. Over the next few days I will finish up the last missing things.

Future

Tobias and I talked about moving the account settings from a dialog to the main window, and removing the apply button, since the process for adding emails addresses needs a separate conformation (because it requires that the user goes to their email inbox). Also, for changing the password the user would have to confirm their changes twice, which doesn’t make much sense. We will probably make some more small changes on the way to make the UX as good as we possibly can.

I’m working on issue #21 and this is my WIP branch.