I finished my master’s degree \o/

In the last couple of months, I was busy writing my thesis to conclude my master’s degree in computer science at the University of Bologna, therefore, I wasn’t much active in the GNOME community I hope that now I have much more time to dedicate to writing software ;).
The title of the thesis is ” Blockchain-based end-to-end encryption for Matrix instant messaging“. I researched an interesting experiment that uses an Ethereum based system to fully end-to-end encrypt a Matrix conversation.

Abstract

Privacy and security in online communication is an important topic today, especially in the context of instant messaging. A lot of progress has been made in recent years to ensure that conversations are secure against attacks by third parties, but privacy from the service provider itself remains difficult. There are a number of solutions offering end-to-end encryption, but most of them rely on a centralized server, proprietary clients, or both.
In order to have fully secure instant messaging conversations, a decentralized and end-to-end encrypted communication protocol is needed. This means there is no single point of control, and each message is encryped directly on the user’s device such that only the recipient can decrypt it.
This work proposes an end-to-end encryption system for the Matrix protocol based on blockchain technology. Matrix is a decentralized protocol and network for real-time communication that is currently mostly used for instant messaging. This protocol was selected because of its versatility and extensibility.
Using the Secret Store feature in OpenEthereum, the proposed system encrypts data using keys stored on the Ethereum blockchain. Access control to the keys is also handled by the Secret Store via a smart contract.
The proposed encryption system has multiple advantages over alternative schemes: The underlying blockchain technology reduces the risk of data loss because of its decentralized and distributed nature. Thanks to the use of smart contracts this system also allows for the creation of an advanced access control system to decryption keys.
In order to test and analyze the proposed design, a reference implementation was created in the form of a library. This library can be used for future research, but also as a building block for different applications to easily implement end-to-end encryption based on blockchain technology.

If you’re interested you can read the full thesis: Blockchain-based end-to-end encryption for Matrix instant messaging.

Joining Purism

This announcement is long overdue, but better late than never :)

About 6 months ago I joined Purism, where I’m working on the Librem 5 phone. I’m in good company, since there are already a number of other fellow GNOME friends on the Librem 5 team, including Adrien Plazas, Tobias Bernard, and Mohammed Sadiq.

The last few months I’ve mainly been working on apps, but also a bit on phone shell (which is also written in GTK). I did a lot of work on Contacts (most of which is upstreamed in 3.36), the SMS and Calls apps, as well as Libhandy. Most recently I’ve been working on adding some initial quick settings toggles to the shell:

It’s been an awesome experience so far, because I get to work on a fully free software phone OS (which we really need, given the sorry state of Google-free Android) while also contributing to GNOME upstream.

Digitizing a analog water meter

For a University project a spent some time working on a project to digitally track the water consumption in my shared flat. Since nowadays everything is about data collection, I wanted to give this idea a shot. In my flat we have a simple analog water meter in my house.

Sadly, my meter is really dirt under the glass and i couldn’t manage to clean it. This will cause problems down the road.

The initial idea was easy, add a webcam on top of the meter and read the number on the upper half it. But I soon realized that the project won’t be that simple. The number shows only the use of 1m^3 (1000 liters), this means that I would have a change only every couple of days, which is useless and boring.  So, I had to read the analog gauges, which show the fraction in 0.0001, 0.001,  0.01 and 0.1 m^3. This discovery blocked me, and I was like “this is way to complicated”.

I have no idea how I found or what reminded me of OpenCV, but that was the solution. OpenCV is an awesome tool for computer vision, it has many features like Facial recognition, Gesture recognition … and also shape recognition. What’s a analog gauge? It’s just a circle with an triangular arrow indicating the value.

Let’s jump in to the project

I’m using a Raspberry Pi 1, a Logitech webcam, a juice bottle and some leds out of a bicycle light.

You need to find a juice bottle which fits nicely over the water meter. Cut of the top and bottom of the bottle and replace one side with a cardboard or wood with a hole in the middle. Attach the webcam centered over the hole and place a led on each side of the webcam to illuminate the water meter (you my need to cover them with paper to reduce reflection on the plastic of the meter).

First step is to set up the Raspberry Pi 1 (it doesn’t have to be a RPI, any computer running linux should work fine). You have to install a Linux Distro on the device, I used Archlinux. You can find a guide to install it on a Raspberry Pi 1 here.

After the initial setup you need to install git, python3 and opencv:
sudo pacman -S python3 git opencv
Clone the needed code to a know location:
git clone https://github.com/jsparber/water-meter-code.git
You need to create a new git repository to store the data and clone it to /home/alarm/water-meter-data/. If you want to use a different name or location you need to modify the name in measure.sh

On the RPI I have a cronjob which runs a script every minute. The script turns on the the led and then takes a picture then it turns the led off again, to save some energy.
With crontab -e you can modify the cron jobs, add * * * * * /home/alarm/code/take_photo.sh to run the take_photo.sh every minute, you may need to adjust the path depending on where you cloned the git repo.

After the picture is taken it calls a second script which then uses OpenCV to read the gauges and it appends the found values to a file which then is pushed to  git repo. I had a issue with the webcam. After some time my script couldn’t access the webcam anymore, I solved it by rebooting the RPI when it wasn’t possible to take a picture. (I did a quick search on the internet, most people solved this issue by changing the cam)

A nice optional feature is the home made switch connected to the RPI on the above picture. The schematics are really simple it’s only a 1KOhm resistor, a transistor and a USB extension cable. The transistor is switched on via the GPIO pin 18 of the Raspberry PI and gives power to the connected USB device. In this case I used it to connect the Leds.

Inside the USB extension cable there should be 4 different colored cables. We need to cut only the red one and connect it the same way as the schematics above show it, where the red_in goes to the male connector and the red_out to the female side of the cable. The GND needs to be connected to the ground pin of the Raspberry Pi, if you need to power something which requires more then 500mA you should connect the ground directly to the power source the same way as you did with the +5V red cable. You need to use the same power source for the switch and the RPI or it may not work.

And now the OpenCv part

First my code finds the circles of the right size on the image, and uses the two most left ones as gauges for 0.1 m^3 and 0.01 m^3 (Sadly since my meter is so dirty I can’t reliably read the other two values).

The input image.
The found circles of the right size

As the second step I create a mask which filters out everything what’s not red (remember the arrows are read). I take the contour of the mask which encloses the center of the circle I want to read. Then it finds the fairest point from the center of the circle which is the tip of the arrow. The software then creates a virtual line between the center and the tip, which is then used to calculate the angle which is basically the value shown on the gauge. The same thing is repeated for the other gauges.

The mask with only red areas showing.
The arrows found on the source image. This lines are used to calculate the angle.

This system sounds extremely simple, but to make everything work well together it isn’t that easy. OpenCV requires a lot of tuning, e.g selecting the right red color so that it detects it well but stays working even with light changes.

Conclusions

I learned a lot during this project, especially about OpenCV which i never used before. Sadly my water meter was really dirty so a couldn’t read all values and get also some wrong readings. So far I didn’t decide for what i want to use the collected data therefore I didn’t spend much time on finding a solution for read errors and problems I have when the gauges make a full turn. A easy solution would be to just keep an internal count of the water. And when we are unsure about a value we can go back to the memorized value.

The final plot can be found here. All values are saved directly without filter this causes the plot to have quite some noise but it allows to change the function used to filter later and adapt it to future needs.

My code is published on github:

Some sources which helped me a lot, many thanks to them:

Barcelona: LAS 2019

This November I was in Barcelona for the Linux App Summit 2019. It was awesome \o/. I really liked that the conference was a joint event by GNOME and KDE, I met so many cool new people. During the conference I volunteered to show the “time left” signs to speakers, and helped out at the registration desk.

Aside from normal conference stuff I also managed to do quite a bit of hacking during the week. I made my first contribution to Gnome Initial Setup, and cleaned up Teleport a bit so I can hopefully get a new release out soon.

I’m bad at taking pictures, so here’s a picture of a tree in the middle of the stairs on the slopes of Mount Montjuic.

Thanks to the GNOME Foundation and Purism for sponsoring my travel and accommodation. The whole event was a lot of fun, so my thanks also go to all the organizers and people who helped make it happen!

GUADEC 2019

This year I did a bit of traveling in Greece before going to GUADEC in Thessaloniki with Tobias, Regina and from Athens Jordan. We started from Kos, a small island close to Turkey. After a couple of days we took a ferry to Athens, where we met up with Jordan, our local tourist guide ;)

Regina, Tobias, and I in front the Parthenon.
Jordan as archaeologist ;))

After visiting Athens and the Acropolis we took upon us the long journey to the oracle of Delphi. Close to Delphi is Mount Parnassus, one of the highest mountains in Greece. Sadly we didn’t manage to climb the peek, but we still had an awesome hike around it.

Mount Parnassus

After the holidays we took the train to Thessaloniki to meet with other Gnomies. Sadly Regina didn’t join us for GUADEC.

So, what have I been up to during GUADEC:

I finally got the icon export feature merged into Icon Preview. I fixed issues with this literally right up until Jakub’s demo, but it was worth it: During his demo the export worked flawlessly. As soon as there’s a new release, you’ll be able to export hicolor and symbolic icons directly from the same template, which improves the icon authoring experience a lot. You can also export a nightly version of the symbolic icon, which is automatically generated with no extra work. We’ve wanted this for a very long time, and it’s great to finally see it come to fruition. Expect Nightly icons in nightly versions of GNOME apps soon!

I also created a new tool (still work in progess) which takes an SVG file and exports a PDF where each page is defined by a rectangle with a specific size and color. Some people (i.e. GNOME designers) like to make their slides in Inkscape (since it’s the only decent free visual editor we have), but since Inkscape doesn’t support multiple pages it’s a real pain to export the slides to PDF. The code isn’t ready to show therefore it isn’t public, yet. I hope to make this into an actual app soon though, to make this workflow feasible for more people.

I also talked to a couple of people about the future of folks (the library powering gnome-contacts) and what possibilities we have to improve the status quo.

Many thanks to everybody involved in organizing GUADEC. It has been a lot of fun, I can’t wait for next year. Special thanks also to the GNOME Foundation for sponsoring my attendance!

Rust Hackfest in Berlin

The Rust hackfest was a few months ago, so apologies for the delayed blog post :)

It was a lot of fun hanging out with all the GNOME people who were in Berlin for this. I unfortunately had some unexpected deadlines from my University.  Therefore I couldn’t hack as much as I wanted but I still got a couple of things done.

I spent most of the time on an example which shows how to implement drag and drop reordering of rows in a GtkListbox which is bound to a ListModel. Sadly I didn’t have the time to fix the few reaming issues (mostly commenting the code), therefore it’s not yet merged. This is the pull request to gtk-rs.

I also worked on Icon Preview to integrate librsvg, so we can have nice nightly icons and the designers can create the hicolor icon in the same file as the symbolic icon. Icon Preview isn’t in Rust, but I filed a couple of issue in librsvg and did some testing there.

Playing “War on Terror” with Regina, Tobias, and grumpy Jordan

Thanks so much to the GNOME Foundation for sponsoring my travel, and to Tobias and Regina for hosting me. Also many thanks to Zeeshan for organizing the hackfest.

Design Tool Hackfest 2019

Last month I was in Berlin for the “Design Tools Hackfest 2019”. This whole thing started during last year’s GUADEC in Almeria, when I started playing around with cairo and librsvg. Because of that I got pulled into some discussions around improving tooling for GNOME designers, especially around icons.

During this hackfest we worked on three main issues:

– How to extract multiple icons contained in a single SVG file as separate SVGs (e.g. the stencil file in adwaita icon theme)
– How to generate Nightly hicolor app icons automatically
– How to export optimized icons directly from Icon Preview

The third point comes more or less for free once the first is done, so we focused on the former two.

Gnome Stencils

Splitting the stencils into different files is currently done with a script which uses Inkscape. This is very very slow because it has to open Inkscape for every single icon.

As far I could find there are no libraries which would allow us to manipulate SVG files in the way we need. Therefore I’m using cario and librsvg to generate the files. This approach may sound crazy, but it works quite well. The basic idea is to render an SVG file into a cairo surface with librsvg and then export the surface via cairo as a new SVG which contains only the part we’re interested in.

Nightly Icons, generated with svago-export

This way we can even use cario masks to automatically render nightly icons, and I started integrating this into Zander’s Icon Preview.

 

Sadly I’m currently quite busy with university, so I didn’t get around to finish and clean up svago-export so far, but if you want to have a look at the experiments I did feel free to do so. Luckily, the semester will soon be over and I will have more free time \o/

Special thanks to Tobias for hosting the event and thanks to the GNOME Foundation for sponsoring my travel.

Another year, another FOSDEM

I have come a long way since my first time at FOSDEM a couple of years ago. The first time it was all new and unknown. I tried to attend as many talks as possible, but could only see half of the talks i wanted to go (amazing how many people there are at FOSDEM). Every year I listened to fewer and fewer talks, because conversations I had outside of talks are so much more fun and appealing. I think the biggest thing which changed is that I’m no longer just a user of free software, but an active contributor.

This year, I spent a lot of the time at the GNOME booth, which is always fun. The GNOME beers event is also awesome, though it was really crowded this year (let’s hope we get a bigger space next year). On Friday we also had a great lunch at a Libanese restaurant. Thanks to Adrien for organizing, and thanks to Purism for offering lunch.

While there, I also did some work on Fractal. I investigated why Fractal uses a lot of CPU and memory and made changes to reduce memory usage.

I stayed at a flat together with Bastian, Niclas, Florian, and Tobias. Thanks a lot for the good time in Brussels, especially to Bastian for organizing the apartment. Special thanks to the GNOME foundation for sponsoring the trip to Brussels.

Fractal Hackfest in Seville

Last month I was in Seville for the second Fractal Hackfest. It was a bit of a different experience for me than the other hackfests I have been too, as I’m a core member to the project now, and I also knew most of the other people already.

My main focus for this hackfest was to push forward the work on the Fractal backend. The backend should handle persistent storage and the preparation of data needed in the UI. Right now many things which should conceptually be in the backend are in the frontend, and can therefore cause UI freezes. Over the past months I have been working hard on refactoring the code so we can just drop in the backend without making many changes to the UI code. So the core of the refactors I did was to make sure that data flows in one direction and we don’t keep different copies of the data around.

Backend

For storage we decided to use SQLite instead of LMDB because we have many relations between the data. The data will be structured into 3 tables: USERS, ROOMS, MESSAGES. This gives us a lot of space to expand and allows us to reference to other data with ease.

The backend will have use a glib based API to communicate with the frontend, and on the other hand it will have a Rust only interface to communicate with fractal-matrix-api.

Community

Lunch at a spanish Pizza place

We also had really awesome community bonding events, and some local newcomers joining the hackfest. Thanks to Daniel for organizing the event and also all locals who helped him.

I fell in love with GAction

A few weeks ago, while working on Fractal, I rediscovered something I had completely forgotten about: GAction. I don’t remember how I came across it, but it was definitely love at second sight, because for the next days I didn’t do much other than refactor Fractal and replace code with Actions wherever I could.

Of course, if you like something a lot you have to be careful to use it only where it’s appropriate. For me it was basically when it’s a user action then it should use GAction, or in other word if it’s a response to an user input, like clicking on a button, then it should be a GAction.

Now, how do you use GAction? Since GAction is only an interface we need to use a class implementing it. The most simple class we can use is of course GSimpleAction as the name already implies. Since I’m writing most of my code in Rust these days, all examples will be in Rust.

// We have to create a SimpleActionGroup
let actions = SimpleActionGroup::new();
// Currently we can't use any Variant type we want, we are limited to the most basic ones like strings, numbers, and bools
let reply = SimpleAction::new("reply", glib::VariantTy::new("s").ok());
// And then we have to add it to the ActionGroup
actions.add_action(&reply);
// The last thing we have to do is to connect to the activate singal
reply.connect_activate(move |action, data| {
//Do whatever you want in here, e.g. open a file dialog
});

The above code shows how an action is created. Let’s see how you can use it.

The first thing we need to do is add the created ActionGroup to a Widget. The action will then be available for all children of that widget, so it’s best to add it to a Container or a Box which allows us to have children.

// Create a Box
let b = gtk::Box::new(gtk::Orientation::Horizontal, 0);
// Add the action group we created before to the widget
box.insert_action_group("something", &actions);

Now the actions are available to all children of the box with the prefix “something”. In our example “something.reply” would be the name of the action. Given a GTKButton you can set the action_name and the target_value of the widget, because it implements GtkActionable. The code basically looks like this:

// Create a new GTK button
let button: gtk::Button = gtk::Button::new();
// We need to use a Variant to store the string we want to set as the target value
let data = glib::Variant::from("some string");
// Set the action name
button.set_action_name("something.reply");
// We also need to set the target value
// If we don't do this, GLib will complain that we didn't call the action with the correct target and won't execute the action
button.set_action_target_value(&data);

You could also set the properties via the .ui file. When you replace the event handlers of clicks with actions you can really streamline your code base.

I showed the basic usage of the GAction in the above examples, but you should definitely have a look at the GAction documentation to see all the potential it has. Also, GPropertyAction is awesome because it allows you to control GObject properties directly via Actions. Sadly, the Rust bindings for it are not in a stable release yet, but they were added to master a couple of months ago.