Igalia is Hiring

Igalia is hiring web browser developers. If you think you’re a good candidate for one of these jobs, you’ll want to fill out the online application accompanying one of the postings. We’d love to hear from you.

We’re especially interested in hiring a browser graphics developer. We realize that not many graphics experts also have experience in web browser development, so it’s OK if you haven’t worked with web browsers before. Low-level Linux graphics experience is the more important qualification for this role.

Igalia is not just a great place to work on cool technical projects like WebKit. It’s also a political and social project: an egalitarian, worker-owned cooperative where everyone has an equal vote in company decisions and receives equal pay. It’s been around for 16 years, so it’s also not a startup. You can work remotely from wherever you happen to be, or from our office in A Coruña, Spain. You won’t have a boss, but you will be expected to work well with your colleagues. It’s not the right fit for everyone, but there’s nowhere I’d rather be.

GNOME 3.26 Core Applications

Last year, I presented the GNOME 3.22 core applications: a recommendation for which GNOME applications have sufficiently-high general appeal that they should be installed out-of-the-box by operating systems that wish to ship the GNOME desktop the way that upstream intends. We received some complaints that various applications were missing from the list, but I was pretty satisfied with the end result. After all, not every high-quality application is going to have wide general appeal, and not every application with general appeal is going to meet our stringent design standards. It was entirely intentional there was not any email client (none met our standards) or chat application (IRC does not have general appeal) included, nor any developer tools (most people aren’t software developers). Our classification was, necessarily, highly-opinionated.

For GNOME 3.24, the list of core applications did not change.

For GNOME 3.26, I’m pleased to announce the addition of three new applications: GNOME Music, GNOME To Do, and Simple Scan. Distributions that choose to follow our recommendation should add these applications to their default install.

Music and To Do have spent the past year maturing. No software is perfect, but these applications are now good enough that it’s time for them to reach a wider audience and hopefully attract some new contributors. In particular, Music has had another major performance improvement that should make it more pleasant to use.

In contrast, Simple Scan has been a mature app for a long time, and has long followed GNOME design guidelines. I’m very happy to announce that development of Simple Scan has moved to GNOME infrastructure. I hope that GNOME will be a good home for Simple Scan for a long time to come.

The full list of core applications for GNOME 3.26 is as follows:

  • Archive Manager (File Roller)
  • Boxes
  • Calculator
  • Calendar (gnome-calendar, not california)
  • Characters (gnome-characters, not gucharmap)
  • Cheese
  • Clocks
  • Contacts
  • Disk Usage Analyzer (Baobab)
  • Disks (gnome-disk-utility)
  • Document Viewer (Evince)
  • Documents
  • Files (Nautilus)
  • Fonts  (gnome-font-viewer)
  • Help (Yelp)
  • Image Viewer (Eye of GNOME)
  • Logs (gnome-logs, not gnome-system-log)
  • Maps
  • Music
  • Photos
  • Screenshot
  • Software
  • Simple Scan
  • System Monitor
  • Terminal
  • Text Editor (gedit)
  • To Do
  • Videos (Totem)
  • Weather
  • Web (Epiphany)

We are now up to 30 core apps! We are not likely to go much higher than this for a while. What changes are likely in the future? My hope is that within the next year we can add Credentials, a replacement for Seahorse (which is not listed above due to quality issues), remove Disk Usage Analyzer and System Monitor in favor of a new Usage app, and hopefully remove Archive Manager in favor of better support for archives in Files. But it’s too early for any of that now. We’ll have to wait and see how things develop!

On Firefox Sync

Epiphany 3.26 is, unfortunately, not going to be packed with cool new features like 3.24 was. We’ve just been too busy working on improving WebKit this cycle. But there is one cool new thing: Firefox Sync support. You can sync bookmarks, history, passwords, and open tabs with other Epiphany instances and as well as both desktop and mobile Firefox. This is already enabled in 3.25.90. Just go to the Sync tab in Preferences and sign in or create your Firefox account there. Please test it out and report bugs now, so we can quash problems you find before 3.26.0 rather than after.

Some thank yous are in order:

  • Thanks to Gabriel Ivascu, for writing all the code.
  • Thanks to Google and Igalia for sponsoring Gabriel’s work.
  • Thanks to Mozilla. This project would never have been possible if Mozilla had not carefully written its terms of service to allow such use.

Go forth and sync!

Endgame for WebKit Woes

In my original blog post On WebKit Security Updates, I identified three separate problems affecting WebKit users on Linux:

  • Distributions were not providing updates for WebKitGTK+. This was the main focus of that post.
  • Distributions were shipping a insecure compatibility package for old, unmaintained WebKitGTK+ 2.4 (“WebKit1”).
  • Distributions were shipping QtWebKit, which was also unmaintained and insecure.

Let’s review these problems one at a time.

Distributions Are Updating WebKitGTK+

Nowadays, most major community distributions are providing regular WebKitGTK+ updates, so this is no longer a problem for the vast majority of Linux users. If you’re using a supported version of Ubuntu (except Ubuntu 14.04), Fedora, or most other mainstream distributions, then you are good to go.

My main concern here is still Debian, but there are reasons to be optimistic. It’s too soon to say what Debian’s policy will be going forward, but I am encouraged that it broke freeze just before the Stretch release to update from WebKitGTK+ 2.14 to 2.16.3. Debian is slow and conservative and so has not yet updated to 2.16.6, which is sad because 2.16.3 is affected by a bug that causes crashes on a huge number of websites, but my understanding is it is likely to be updated in the near future. I’m not sure if Debian will update to 2.18 or not. We’ll have to wait and see.

openSUSE is another holdout. The latest stable version of openSUSE Leap, 42.3, is currently shipping WebKitGTK+ 2.12.5. That is disappointing.

Most other major distributions seem to be current.

Distributions Are Removing WebKitGTK+ 2.4

WebKitGTK+ 2.4 (often informally referred to as “WebKit1”) was the next problem. Tons of desktop applications depended on this old, insecure version of WebKitGTK+, and due to large API changes, upgrading applications was not going to be easy. But this transition is going much smoother and much faster than I expected. Several distributions, including Debian, Fedora, and Arch, have recently removed their compatibility packages. There will be no WebKitGTK+ 2.4 in Debian 10 (Buster) or Fedora 27 (scheduled for release this October). Most noteworthy applications have either ported to modern WebKitGTK+, or have configure flags to disable use of WebKitGTK+. In some cases, such as GnuCash in Fedora, WebKitGTK+ 2.4 is being bundled as part of the application build process. But more often, applications that have not yet ported simply no longer work or have been removed from these distributions.

Soon, users will no longer need to worry that a huge amount of WebKitGTK+ applications are not receiving security updates. That leaves one more problem….

QtWebKit is Back

Upstream QtWebKit has not been receiving security updates for the past four years or thereabouts, since it was abandoned by the Qt project. That is still the status quo for most distributions, but Arch and Fedora have recently switched to Konstantin Tokarev’s fork of QtWebKit, which is based on WebKitGTK+ 2.12. (Thank you Konstantin!) If you are using any supported version of Fedora, you should already have been switched to this fork. I am hopeful that the fork will be rebased on WebKitGTK+ 2.16 or 2.18 in the near future, to bring it current on security updates, but in the meantime, being a year and a half behind is an awful lot better than being four years behind. Now that Arch and Fedora have led the way, other distributions should find little trouble in making the switch to Konstantin’s QtWebKit. It would be a disservice to users to continue shipping the upstream version.

So That’s Cool

Things are better. Some distributions, notably Arch and Fedora, have resolved all of the above problems (or will in the very near future). Yay!

Modifying hidden settings in Epiphany 3.24

We’re just one short month away from releasing Epiphany 3.26, but this is not a post about that. Turns out there is some confusion about how to edit hidden settings in Epiphany 3.24. Many users previously relied on the dconf-editor tool to tweak hidden settings like the user agent or minimum font size, but this no longer works in 3.24. What gives?

The problem is that these settings can now be configured separately for your main browsing instance and for each web app. This gives you a lot more flexibility, but it does make it harder to change the settings because dconf-editor will not work anymore. The technical problem is that dconf-editor does not support relocatable settings schemas: settings definitions that are reused in many different places. So you will unfortunately have to use the command line to change these settings now. For example:

# Old command, *this no longer works*
$ gsettings set org.gnome.Epiphany.web user-agent 'Mozilla/5.0'

# Replacement command
$ gsettings set org.gnome.Epiphany.web:/org/gnome/epiphany/web/ user-agent 'Mozilla/5.0'

Changing a global setting like this will also affect newly-created web apps, but not existing web apps.

Debian Stretch ships latest WebKitGTK+

I’ll keep this update short. Debian has decided to ship the latest version of WebKitGTK+, 2.16.3, in its upcoming Stretch release. Since Debian was the last major distribution holding out on providing WebKit security updates, this is a big deal. Huge thanks to Jeremy Bicha for making this possible.

The bad news is that Debian is still considering whether or not to provide periodic security updates after the release, so there might not be any. But maybe there will be. We will have to wait and see. At least releasing with the latest version is a big step in the right direction.

How I Learned to (Mostly) Love Private Internet Access

TL;DR: I’ve renewed my subscription to Private Internet Access, and intend to continue using the service indefinitely.

This is the third and final blog post in my series on Private Internet Access. Part One lists the different problems I encountered when trying to use Private Internet Access, and Part Two discusses how I solved most of them. When Part Two was published, my remaining unsolved problems were (a) extreme difficulty checking mail in Evolution, (b) my first attempt to connect always failed, and (c) I was blocked from freenode. A day after publishing the second post, I updated it to discuss how to get the first connection attempt to work (save your password system-wide so it’s accessible by the login screen… seems obvious in retrospect).

So what did I do about email and freenode?

Email

I’m really happy with my solution for email. The problem was that I experienced a very high number of timeout errors sending and receiving messages when using Private Internet Access, far more than when not using it. A PR representative from Private Internet Access told me I needed to ask them to whitelist our mail server for SMTP, but I knew that wasn’t the problem because it worked OK sometimes, and I was having trouble with IMAP too. Everything email-related was just so much slower when using Private Internet Access.

My solution was to uninstall Evolution and install Geary instead. I now wish I had done this a long time ago. Geary has many shortcomings and significant room for improvement, but I’ve never been more pleased with a mail client. With Geary, reading my mail is no longer painful. Whereas Evolution takes several seconds to load every individual message, and often times out and fails, even when not using Private Internet Access, Geary takes a few seconds to load an entire conversation, which speeds things up tremendously. Conversation view is killer, a real must-have for a mail client. More importantly, timeouts and error messages are extremely rare with Geary, even when using Private Internet Access. Probably the difference is that Geary just waits a lot longer before timing out. I did experience one day shortly after switching to Geary where I was unable to send any mail from my Igalia account, which at the time I attributed to Private Internet Access. However, I’ve had no trouble since then, so I think this was  just an intermittent problem.  Geary also has a much slicker user interface than Evolution. I’m not comfortable saying that Geary is going to be the future of mail for GNOME, since there is no question that Evolution is a far more capable client right now, but I’m very pleased with Geary and am looking forward to future development.

freenode

I’m really unhappy about my solution for freenode. If you use an  IRC client that has good support for NickServ or SASL authentication, then apparently there is nothing you need to do to access freenode besides configure that. However, neither Empathy nor Polari qualify here, and those are the only IRC clients that are interesting to me personally. With a little experimentation (and some help from Florian), I found both clients could be configured to authenticate with NickServ automatically. However, there’s no way to avoid being pestered with a private message in GNOME Shell from NickServ every time I connect, with the accompanying chat box to type my response. The Telepathy integration in GNOME Shell needs some serious work.

So I went a couple weeks where I rarely ever logged into freenode, using only the KiwiIRC web client when I needed to join for something specific, like for a meeting or to contact a specific person. Now, KiwiIRC is actually pretty nice and functional, but a web client doesn’t really meet my needs for daily use. In the end, I settled on connecting to freenode via Igalia’s Matrix server. Yes, I’m using Riot and, yes, that’s another web client, but I have to use it anyway, so it’s no difference to me.  Now, Matrix seems to be a really nice chat protocol, and Riot is at least decent as a Matrix client, but it is an extremely terrible IRC client. For one, there’s no way to tell who is online outside your own Matrix server. Seriously. (Why so many people are using it to access GIMPNet, I have no clue.) So I still log in to KiwiIRC whenever I need to check if someone is online on freenode, while continuing to connect to GIMPNet from Empathy, because — and I never thought I’d say this — Empathy at least works properly. This is a very poor solution, but it’s a worthwhile tradeoff for me to be able to use Private Internet Access. It also allows me to avoid the disastrous non-bug where Matrix silently drops any private messages that a non-authenticated IRC user sends to a Matrix bridge user on freenode. (Silently!) I’m told this is an intentional anti-spam feature, but I think it’s totally unacceptable. It just sounds to me like maybe Matrix should not be in the business of bridging to IRC at all, if it can’t figure out how to handle PMs. I’m not sure I’ve ever experienced any PM spam. But anyway, this is supposed to be a blog post about Private Internet Access, not a rant about Matrix’s IRC bridge, so that’s enough complaining about that.

Conspiracy Theory

So besides the fact that IRC is terrible, I’m pretty satisfied with Private Internet Access. You have to trust it, though. It’s not often that relatively small companies decide to spend tens of thousands of dollars sponsoring free software projects like GNOME. (Private Internet Access does that!) For all we know, it could be run by the NSA, seeking to gobble up the web browsing history of people who think they have something to hide, and donating to free software because it knows that free software users will recommend the service to more people. That’s a totally-unsubstantiated claim that I just made up and for which I have zero evidence to support, but I don’t know, and you don’t know, and that’s the point. You have to trust it. Or at least, you have to trust it relative to the level of trust you have in your ISP. But you probably shouldn’t trust your ISP, at least not if it’s a national company, so that makes Private Internet Access an easy choice for me.

Update: Read the first comment below. What on Earth is going on?

On reCAPTCHA Dread

When did reCAPTCHA become so maddeningly difficult and time-consuming?

I wanted to read Matthew Garrett’s post on Intel’s remote AMT vulnerability, but since I’m using Private Internet Access, Cloudflare has gated it behind reCAPTCHA. reCAPTCHA is much, much harder than it used to be. Although there seem to be a couple of other variants, nowadays you’re generally expected to identify squares that contain street signs and squares that contain mountains. Now either the answer key is regularly wrong, or I just don’t know what street signs and mountains are. You’d think the former… but there actually is a good degree of ambiguity in selecting which squares to tag.  Do I only tag all the squares that contain the signage-portion of the sign, or do I also tag the squares containing the signpost? (The former seems to work better, in my experience.) What if only a little bit of the sign extends into a particular square? (Jury’s out.) What if there are very distant signs in the background of the image, with many big signs in the foreground: should the distant signs be tagged too? And what constitutes a mountain anyway? Most of the “mountains” I see in the reCAPTCHA images look more like impressive hills to me. My guess is that reCAPTCHA wants me to tag any bit of elevated land as a mountain, but who knows, really.

Worse, once you tag some squares as signs or mountains, more squares appear in their places, making the captcha seem almost never-ending: you never know when you’ll finish. I dread each time new squares appear. I regularly find myself taking much too long to solve the captcha, often as much time as I would actually spend reading the gated content. Once I finish, I await the answer… “try again.” Which squares did I get wrong, Google? There’s no way to know. Back to start.

Today, for the first time that I can recall, I gave up after I was told to Try Again a few too many times in a row. I refreshed the page, thinking to try the captcha again… no luck this time, either. So congratulations, Google, on designing a usability nightmare captcha that can keep humans out. I know it has to be hard enough to make it difficult to solve with computer vision, but if humans can’t pass it, your captcha has failed. And so I am stuck reading Matthew’s article in Liferea.

Remember the days when all you had to do was spell out two words?

More On Private Internet Access

A few quick follow-up thoughts from my original review. First, problems I haven’t solved yet:

  • I forgot an important problem in my first blog: email. Evolution is borderline unusable with PIA. My personal GMail account usually works reliably, but my Google Apps school GMail account (which you’d think would function the same) and my Igalia email both time out with the error “Source doesn’t support prompt for credentials”. That’s Evolution’s generic error that it throws up whenever the mail server is taking too long to respond. So what’s going on here? I can check my email via webmail as a workaround in the meantime, but this is really terrible.
  • Still no solution for the first attempt to connect always failing. That’s really annoying! I was expecting some insight (or at least guesses) as to what might be going wrong here, but nobody has suggested anything about this yet. Update: The problem is that I had selected “Make available to other users” but “Store the password only for this user”, which results in the first attempt to connect always failing, because it’s performed by the gdm user. The fix is to store the password for all users.

Some solutions and answers to problems from my original post:

  • Jonh Wendell suggested using TCP instead of UDP to connect to PIA. I’ve been trying this and so far have not noticed a single instance of connection loss. So I think my biggest problem has been solved. Yay!
  • Dan LaManna posted a link to vpnfailsafe. I’m probably not going to use this since it’s a long shell script that I don’t understand, and since my connection drop problems seem to be solved now that I’ve switched to TCP, but it looks like it’d probably be a good solution to its problem. Real shame this is not built in to NetworkManager already.
  • Christel Dahlskjaer has confirmed that freenode requires NickServ/SASL authentication to use via PIA. This isn’t acceptable for me, since Empathy can’t handle it well, so I’m probably just going to stop using freenode for the most part. The only room I was ever really active in was #webkitgtk+, but in practice our use of that room is basically redundant with #epiphany on GIMPNet (where you’ll still find me, and which would be a better location for a WebKitGTK+ channel anyway), so I don’t think I’ll miss it. I’ve been looking to reduce the number of IRC rooms I join for a long time anyway. The only thing I really need freenode for is Fedora Workstation meetings, which I can attend via a web gateway. (Update: I realized that I am going to miss #webkit as well. Hmm, this could be a problem….)

So my biggest issue now is that I can’t use my email. That’s pretty surprising, as I wouldn’t think using a VPN would make any difference for that. I don’t actually care about my Google Apps account, but I need to be able to read my Igalia mail in Evolution. (Note: My actual IP seems to leak in my email headers, but I don’t care. My name is on my emails anyway. I just care that it works.)

On Private Internet Access

I’m soon going to be moving to Charter Communications territory, but I don’t trust Charter and don’t want it to keep records of all the websites that I visit.  The natural solution is to use a VPN, and the natural first choice is Private Internet Access, since it’s a huge financial supporter of GNOME, and I haven’t heard anybody complain about problems with using it. This will be a short review of my experience.

The service is not free. That’s actually good: it means I’m the customer, not the product. Cost is $40 per year if you pay a year in advance, but you should probably start with the $7/month plan until you’re sure you’re happy with the service and will be keeping it long-term. Anyway, this is a pretty reasonable price that I’m happy to pay.

The website is fairly good. It makes it easy to buy or discontinue service, so there are no pricing surprises, and there’s a pretty good library of support documentation. Unfortunately some of the claims on the website seem to be — arguably — borderline deceptive. A VPN service provides excellent anonymity against your ISP, but relying on a VPN would be a pretty bad idea if your adversary is the government (it can perform a traffic correlation attack) or advertising companies (they know your screen resolution, the performance characteristics of your graphics card, and until recently the rate your battery drains…). But my adversary is going to be Charter Communications, so a VPN is the perfect solution for me. If you need real anonymity, you absolutely must use the Tor Browser Bundle, but that’s going to make your life harder, and I don’t want my life to be harder, so I’ll stick with a VPN.

Private Internet Access provides an Ubuntu app, but I’m going to ignore that because (a) I use Fedora, not Ubuntu, and (b) why on Earth would you want a separate desktop app for your VPN when OpenVPN integration is already built-in on Ubuntu and all modern Linux desktops? Unfortunately the documentation provided by Private Internet Access is not really sufficient — they have a script to set it up automatically, but it’s really designed for Ubuntu and doesn’t work on Fedora — so configuration was slightly challenging.  I wound up following instructions on some third-party website, which I have long since forgotten. There are many third-party resources for how to configure PIA on Linux, which you might think is good but actually indicates a problem with the official documentation in my opinion. So there is some room for improvement here. PIA should ditch the pointless desktop app and improve its documentation for configuring OpenVPN via NetworkManager. (Update: After publishing this post, I discovered this article. Seems the installation script now supports for Fedora/RHEL and Arch Linux. So my claim that it only works on Ubuntu is outdated.) But anyway, once you get it configured properly with NetworkManager, it works: no need to install anything (besides the OpenVPN certificate, of course).

Well, it mostly works. Now, I have two main requirements to ensure that Charter can’t keep records of the websites I’m visiting:

  • NetworkManager must autoconnect to the VPN, so I don’t have to do it manually.
  • NetworkManager must reconnect to the VPN service if connection drops, and must never send any data if the VPN is off.

The first requirement was hard to solve, and I still don’t have it working perfectly. There is no GUI configuration option for this in gnome-control-center, but I eventually found it in nm-connection-editor: you have to edit your normal non-VPN connection, which has a preference to select a VPN to connect to automatically. So we should improve that in gnome-control-center. Unfortunately, it doesn’t work at all the first time your computer connects to the internet after it’s booted. Each time I boot my computer, I’m greeted with a Connection Failed notification on the login screen. This is probably a NetworkManager bug. Anyway, after logging in, I just have to manually connect once, then it works.

As for the next requirement, I’ve given up. My PIA connection is routinely lost about once every 30-45 minutes, usually when watching YouTube or otherwise using a lot of data. This is most likely a problem with PIA’s service, but I don’t know that: it could just as well be my current ISP cutting the connection, or maybe even some client-side NetworkManager bug. Anyway, I could live with brief connection interruptions, but when this happens, I lose connection entirely for about a minute — too long — and then the VPN times out and NetworkManager switches back to sending all the data outside the VPN. That’s totally unacceptable. To be clear, sending data outside the VPN is surely a NetworkManager problem, not a PIA problem, but it needs to be fixed for me to be comfortable using PIA. I see some discussion about that on this third-party GitHub issue, but the “solution” there is to stop using NetworkManager, which I’m not going to do. This is probably one of the reasons why PIA provides a desktop app — I think the PIA app doesn’t suffer from this issue? — but like I said, I’m not going to use a third-party OpenVPN app instead of the undoubtedly-nicer support that’s built in to GNOME.

Another problem is that I can’t connect to Freenode when I’m using the VPN. GIMPNet works fine, so it’s not a problem with IRC in general: Freenode is specifically blocking Private Internet Access users. This seems very strange, since Freenode has a bunch of prominent advertising for PIA all over its website. I could understand blocking PIA if there are too many users abusing it, but not if you’re going to simultaneously advertise it.

I also cannot access Igalia’s SIP service when using PIA. I need that too, but that’s probably something we have to fix on our end.

So I’m not sure what to do now. We have two NetworkManager bugs and a problem with Freenode. Eventually I’ll drop Empathy in favor of Matrix or some other IRC client where registering with NickServ is not a terrible mistake (presumably they’re only blocking unregistered users?), so the Freenode issue seems less-important. I think I’d be willing to just stop visiting Freenode if required to use PIA, anyway. But those NetworkManager issues are blockers to me. With those unfixed, I’m not sure if I’m going to renew my PIA subscription or not. I would definitely renew if someone were to fix those two issues. The ideal solution would be for PIA to adopt NetworkManager’s OpenVPN plugin and ensure it gets cared for, but if not, maybe someone else will fix it?

Update: See part two for how to solve some of these problems.